types: cut off display after 10000 nodes

We have logic to output `...` when a type gets too deep and to stop
recursing at that point. Most notably this is useful when outputting
an occurs-check-failing type. But the max depth is 64, so you can make a
type of size 2^64 without hitting this limit, which will still blow out
your memory and effectively stall.

This can create pretty confusing symptoms when fuzzing for pathological
programs -- you will get an apparent stall with no output, and no matter
how many debug printlns you add it will appear that they all execute
before the stall -- and it turns out that in fact all your code was
working correctly up to the point of the final panic where it tries to
*display* an occurs-check error.

Cut off the display at 10000 nodes. If somebody really needs to output a
text representation of such a thing they can write their own logic.

Author: apoelstra

src/types/mod.rs

@@ -368,12 +368,17 @@ impl Type {
 }
 
 const MAX_DISPLAY_DEPTH: usize = 64;
+const MAX_DISPLAY_LENGTH: usize = 10000;
 
 impl fmt::Debug for Type {
     fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
         for data in (&self.ctx, self.inner.bound.root())
             .verbose_pre_order_iter::<NoSharing>(Some(MAX_DISPLAY_DEPTH))
         {
+            if data.index > MAX_DISPLAY_LENGTH {
+                write!(f, "... [truncated type after {} nodes]", MAX_DISPLAY_LENGTH)?;
+                return Ok(());
+            }
             if data.depth == MAX_DISPLAY_DEPTH {
                 if data.n_children_yielded == 0 {
                     f.write_str("...")?;
@@ -407,6 +412,10 @@ impl fmt::Display for Type {
         for data in (&self.ctx, self.inner.bound.root())
             .verbose_pre_order_iter::<NoSharing>(Some(MAX_DISPLAY_DEPTH))
         {
+            if data.index > MAX_DISPLAY_LENGTH {
+                write!(f, "... [truncated type after {} nodes]", MAX_DISPLAY_LENGTH)?;
+                return Ok(());
+            }
             if data.depth == MAX_DISPLAY_DEPTH {
                 if data.n_children_yielded == 0 {
                     f.write_str("...")?;