Ticket #97 (closed defect: fixed)

Opened 5 months ago

Last modified 3 months ago

Consistent heap overflows during parsing on some configurations

Reported by: jmaessen Assigned to: jdn
Priority: critical Milestone:
Component: interpreter Version:
Keywords: Cc:

Description

I just upgraded to Apple's 64-bit Java 1.6 on OS X. I can consistently cause a heap overflow on parsing if I do the following:

$FORTRESS_HOME/ProjectFortress/ant cleanCache
$FORTRESS_HOME/bin/fortress [anything here]

The interesting workaround is to ant test after cleaning the cache. Sadly, this is not always an acceptable solution when you're in a compile-edit-run-debug cycle and have a known broken program you're trying to turn around fast.

The fact that we have heap overflows during parsing at all is a big red flag. I'm baffled; if it were a stack overflow I'd guess a right recursion in the grammar where there ought to be a * instead (in general * is preferred to recursion in any PEG). Especially suspicious is the fact that this only occurs if the libraries are unparsed. It looks like we're not traversing the libraries in a sensible order during pre-parse. I'd love it if someone who understood this code would have a look and fix it. I'm guessing Janus is the obvious candidate, but Sukyoung or David may be able to shed light.

Most of our user base is unlikely to run ant test after every cache clean (and they shouldn't have to).

Change History

05/05/08 23:42:02 changed by jdn

  • status changed from new to assigned.

05/06/08 04:59:12 changed by jmaessen

Here's a stack trace for source:trunk/ProjectFortress/hello.fss : $ ../bin/fortress hello.fss guessing FORTRESS_HOME=../bin/.. Exception in thread "main" java.lang.OutOfMemoryError?: Java heap space

at xtc.parser.ParserBase?.growBy(ParserBase?.java:199) at xtc.parser.ParserBase?.column(ParserBase?.java:226) at com.sun.fortress.parser.Fortress.pWhitespace(Fortress.java:47642) at com.sun.fortress.parser.Fortress.pw(Fortress.java:48081) at com.sun.fortress.parser.Fortress.pParameter$Params$$Star1$1(Fortress.java:6806) at com.sun.fortress.parser.Fortress.pParameter$Params$$Star1(Fortress.java:6779) at com.sun.fortress.parser.Fortress.pParameter$Params$1(Fortress.java:6671) at com.sun.fortress.parser.Fortress.pParameter$Params(Fortress.java:6657) at com.sun.fortress.parser.Fortress.pParameter$ValParam?$1(Fortress.java:6524) at com.sun.fortress.parser.Fortress.pParameter$ValParam?(Fortress.java:6487) at com.sun.fortress.parser.Fortress.pNamedFnHeaderFront$1(Fortress.java:6390) at com.sun.fortress.parser.Fortress.pNamedFnHeaderFront(Fortress.java:6348) at com.sun.fortress.parser.Fortress.pLocalFnDecl$1(Fortress.java:40016) at com.sun.fortress.parser.Fortress.pLocalFnDecl(Fortress.java:39988) at com.sun.fortress.parser.Fortress.pLocalVarFnDecl(Fortress.java:39925) at com.sun.fortress.parser.Fortress.pBlockElem(Fortress.java:39851) at com.sun.fortress.parser.Fortress.pBlockElemCollection$1(Fortress.java:39452) at com.sun.fortress.parser.Fortress.pBlockElemCollection(Fortress.java:39437) at com.sun.fortress.parser.Fortress.pBlockElems$1(Fortress.java:39410) at com.sun.fortress.parser.Fortress.pBlockElems(Fortress.java:39398) at com.sun.fortress.parser.Fortress.pTypecaseClause$1(Fortress.java:34511) at com.sun.fortress.parser.Fortress.pTypecaseClause(Fortress.java:34482) at com.sun.fortress.parser.Fortress.pTypecaseClauses(Fortress.java:34431) at com.sun.fortress.parser.Fortress.pDelimitedExpr$1(Fortress.java:32973) at com.sun.fortress.parser.Fortress.pDelimitedExpr(Fortress.java:32341) at com.sun.fortress.parser.Fortress.pNoNewlineExpr$ExprFront?(Fortress.java:35911) at com.sun.fortress.parser.Fortress.pNoNewlineExpr$Expr$1(Fortress.java:35799) at com.sun.fortress.parser.Fortress.pNoNewlineExpr$Expr(Fortress.java:35787) at com.sun.fortress.parser.Fortress.pNoNewlineExpr$1(Fortress.java:37550) at com.sun.fortress.parser.Fortress.pNoNewlineExpr(Fortress.java:37538) at com.sun.fortress.parser.Fortress.pBlockElem(Fortress.java:39861) at com.sun.fortress.parser.Fortress.pBlockElemCollection$1(Fortress.java:39452)

05/06/08 04:59:44 changed by jmaessen

Actually, here's that trace with better quoting:

$ ../bin/fortress hello.fss
guessing FORTRESS_HOME=../bin/..
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at xtc.parser.ParserBase.growBy(ParserBase.java:199)
	at xtc.parser.ParserBase.column(ParserBase.java:226)
	at com.sun.fortress.parser.Fortress.pWhitespace(Fortress.java:47642)
	at com.sun.fortress.parser.Fortress.pw(Fortress.java:48081)
	at com.sun.fortress.parser.Fortress.pParameter$Params$$Star1$1(Fortress.java:6806)
	at com.sun.fortress.parser.Fortress.pParameter$Params$$Star1(Fortress.java:6779)
	at com.sun.fortress.parser.Fortress.pParameter$Params$1(Fortress.java:6671)
	at com.sun.fortress.parser.Fortress.pParameter$Params(Fortress.java:6657)
	at com.sun.fortress.parser.Fortress.pParameter$ValParam$1(Fortress.java:6524)
	at com.sun.fortress.parser.Fortress.pParameter$ValParam(Fortress.java:6487)
	at com.sun.fortress.parser.Fortress.pNamedFnHeaderFront$1(Fortress.java:6390)
	at com.sun.fortress.parser.Fortress.pNamedFnHeaderFront(Fortress.java:6348)
	at com.sun.fortress.parser.Fortress.pLocalFnDecl$1(Fortress.java:40016)
	at com.sun.fortress.parser.Fortress.pLocalFnDecl(Fortress.java:39988)
	at com.sun.fortress.parser.Fortress.pLocalVarFnDecl(Fortress.java:39925)
	at com.sun.fortress.parser.Fortress.pBlockElem(Fortress.java:39851)
	at com.sun.fortress.parser.Fortress.pBlockElemCollection$1(Fortress.java:39452)
	at com.sun.fortress.parser.Fortress.pBlockElemCollection(Fortress.java:39437)
	at com.sun.fortress.parser.Fortress.pBlockElems$1(Fortress.java:39410)
	at com.sun.fortress.parser.Fortress.pBlockElems(Fortress.java:39398)
	at com.sun.fortress.parser.Fortress.pTypecaseClause$1(Fortress.java:34511)
	at com.sun.fortress.parser.Fortress.pTypecaseClause(Fortress.java:34482)
	at com.sun.fortress.parser.Fortress.pTypecaseClauses(Fortress.java:34431)
	at com.sun.fortress.parser.Fortress.pDelimitedExpr$1(Fortress.java:32973)
	at com.sun.fortress.parser.Fortress.pDelimitedExpr(Fortress.java:32341)
	at com.sun.fortress.parser.Fortress.pNoNewlineExpr$ExprFront(Fortress.java:35911)
	at com.sun.fortress.parser.Fortress.pNoNewlineExpr$Expr$1(Fortress.java:35799)
	at com.sun.fortress.parser.Fortress.pNoNewlineExpr$Expr(Fortress.java:35787)
	at com.sun.fortress.parser.Fortress.pNoNewlineExpr$1(Fortress.java:37550)
	at com.sun.fortress.parser.Fortress.pNoNewlineExpr(Fortress.java:37538)
	at com.sun.fortress.parser.Fortress.pBlockElem(Fortress.java:39861)
	at com.sun.fortress.parser.Fortress.pBlockElemCollection$1(Fortress.java:39452)

06/04/08 08:53:01 changed by jmaessen

  • summary changed from Consistent stack overflows on 64-bit 1.6 on OS X to Consistent heap overflows during parsing on some configurations.

Jon R. is encountering these overflows elsewhere as well. Temporary fix is to increase memory. This ticket should be left in place pending an actual fix to the parser to avoid these problems. Note that there are only 17688K of tfs and tfi files, generated from 3476K of fss and fsi files, so the amount of storage required ought to be much smaller.

A bunch of work has gone on to remove recursive traversal from the parsing path (courtesy Janus et al.) so that we're processing one or two files at a time rather than chasing nested imports on the fly. That doesn't seem to have actually cleared up the memory usage problems. It'd probably be worth profiling the heap to figure out where the storage is going before attempting further fixes (though in the latter case the attempted fix also involved a much-needed code cleanup, which is a good thing on its own---we should do those whether or not they fix bugs).

06/12/08 14:22:50 changed by LouisCutrona

I just got r1885 and this problem appeared on both my 1/2 Gb machine and my 2Gb machine. I dropped back to r1819, which I was using before, and the problem does not occur.

Actual console output example (different fss program, slightly different output from what was already posted):

OS: Windows XP with up-to-date patches. Java: JDK1.6.0_06. Ant: 1.7.0 Cygwin DLL product version: 1.5.25-cr-0x5f1 Fortress r1885 obtained via svn checkout.

LJC@Aphrodite /cygdrive/c/temp
$ fortress compile IzhikevichB.fss

LJC@Aphrodite /cygdrive/c/temp
$ fortress IzhikevichB.fss
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at xtc.parser.ParserBase.growBy(ParserBase.java:199)
        at xtc.parser.ParserBase.column(ParserBase.java:226)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48193)
        at com.sun.fortress.parser.Fortress.pid$$Star1$1(Fortress.java:48212)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48196)
        at com.sun.fortress.parser.Fortress.pid$$Star1$1(Fortress.java:48212)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48196)
        at com.sun.fortress.parser.Fortress.pid$$Star1$1(Fortress.java:48212)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48196)
        at com.sun.fortress.parser.Fortress.pid$$Star1$1(Fortress.java:48212)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48196)
        at com.sun.fortress.parser.Fortress.pid$$Star1$1(Fortress.java:48212)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48196)
        at com.sun.fortress.parser.Fortress.pid$$Star1$1(Fortress.java:48212)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48196)
        at com.sun.fortress.parser.Fortress.pid$$Star1$1(Fortress.java:48212)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48196)
        at com.sun.fortress.parser.Fortress.pid$$Star1$1(Fortress.java:48212)
        at com.sun.fortress.parser.Fortress.pid$$Star1(Fortress.java:48196)
        at com.sun.fortress.parser.Fortress.pid$$Choice1(Fortress.java:48249)
        at com.sun.fortress.parser.Fortress.pid$1(Fortress.java:48165)
        at com.sun.fortress.parser.Fortress.pid(Fortress.java:48153)
        at com.sun.fortress.parser.Fortress.pIdText(Fortress.java:49683)
        at com.sun.fortress.parser.Fortress.pId$1(Fortress.java:49735)
        at com.sun.fortress.parser.Fortress.pId(Fortress.java:49713)
        at com.sun.fortress.parser.Fortress.pNamedFnHeaderFront$1(Fortress.java:8134)
        at com.sun.fortress.parser.Fortress.pNamedFnHeaderFront(Fortress.java:8120)
        at com.sun.fortress.parser.Fortress.pLocalFnDecl$1(Fortress.java:47254)
        at com.sun.fortress.parser.Fortress.pLocalFnDecl(Fortress.java:47226)
        at com.sun.fortress.parser.Fortress.pLocalVarFnDecl(Fortress.java:47163)
        at com.sun.fortress.parser.Fortress.pBlockElem(Fortress.java:47089)
        at com.sun.fortress.parser.Fortress.pBlockElemCollection$1(Fortress.java:46690)

06/13/08 10:36:58 changed by mspiegel

I have seen the same behavior on Vista running natively without Cygwin. As a temporary workaround, I'm using "-Xms128m -Xmx512m" as arguments to java inside the script bin/fortress (actually I modified bin/fortress.bat since I'm not using Cygwin). Where's all our memory going?

07/14/08 18:57:30 changed by jmaessen

Closing. Thanks to Ryan's grammar profiling tools and Sukyoung's changes to the grammar to make more stuff transient, this is no longer a problem; space usage during parsing has dropped precipitously. Still more tuning possible, but looking very good.

07/15/08 13:48:11 changed by jmaessen

  • status changed from assigned to closed.
  • resolution set to fixed.