Thursday, January 15, 2009

Info tables

The info tables are getting broken for some reason. The Cmm code has:

sEl_ret()
{ [const Main.$wf_srt-sEl_info;, const 1;, const 2228231;]
}

-fvia-c gives:

.text
.align 4
.long Main_zdwf_srt - sEl_info
.long 1
.long 2228231
sEl_info:
....

but -fasm gives:

.text
.align 4
.long Main_zdwf_srt+0 <---------- lost sE1_info
.long 1
.long 2228231
sEl_info:
....

some time later..

#if sparc_TARGET_ARCH
-- ToDo: This should really be fixed in the PIC support, but only
-- print a for now.
pprImm (ImmConstantDiff a b) = pprImm a
#else
pprImm (ImmConstantDiff a b) = pprImm a <> char '-'
<> lparen <> pprImm b <> rparen
#endif

!!!! .... sigh.

Fixed that, but it still doesn't work. Here's the code for a whole closure:

    .data                        <-------------------------------- data
.align 8
.global Main_a_srt
Main_a_srt:
.long Main_lvl_closure
.long base_GHCziHandle_stdout_closure
.long base_GHCziIO_a28_closure
.data
.align 8
.global Main_a_closure
Main_a_closure:
.long Main_a_info
.long 0
.text <-------------------------------- text
.align 4
.long Main_a_srt-(Main_a_info)+0 <-------- offset text to data
.long 196609
.long 0
.long 983047
.global Main_a_info
Main_a_info:
.LcGp:
sethi %hi(base_GHCziHandle_stdout_closure),%l2
or %l2,%lo(base_GHCziHandle_stdout_closure),%l2
sethi %hi(Main_lvl_closure),%l3
or %l3,%lo(Main_lvl_closure),%l3
call base_GHCziIO_a28_info,0
nop

SRTs (Static Resource Table?) are supposibly used for garbage collecting CAFs, but the GHC commentary page on them seems out of date, or missing. In any event, the assembler can't make an offset between labels in .text and .data segments. In some architectures .text and .data use entirely separate address spaces. This probably got broken in a previous GHC release when info tables were moved to be next to the code. Checking against x86 reveals it does the same thing, but the x86 assembler is ok with cross segment offsets.

I ended up just changing the pretty printer so it prints out ReadOnlyData segments as .text, which is a bit nasty. I'll be able to handle this in a nicer way when the sparc NCG is factored out into its own set of modules.

Win!

56 expected passes
1 expected failures
0 unexpected passes
3 unexpected failures

Unexpected failures:
2080(optasm) -- wrong output
cg015(optasm) -- unknown unary match op
cg054(optasm) -- genSwitch

That seems to have fixed the seg faulting ones.

Working on 2080.hs

-- cmm code is
_sFv::I32 = %MO_UU_Conv_W8_W32(I8[R2]); <--- load unsigned
_sFx::I32 = _sFv::I32;
_cG7::I32 = %MO_S_Le_W32(_sFx::I32, 127);
if (_cG7::I32 >= 1) goto cGa;

-- -fvia-c gives:
ldub [%l2], %g1 <--- load unsigned
cmp %g1, 127
ble,pt %icc, .LL5
sethi %hi(ghczmprim_GHCziBool_True_closure), %g1
...

-- -fasm gives:
ldsb [%l2],%l0 <--- load signed
srl %l0,0,%l0 <--- nop
cmp %l0,127
ble .LcGa
nop
...

This was an easy operand format problem. However, I did notice that the Cmm code only ever loads unsigned data, like I8[R2]. If you were going to load signed data it would be better to use the sparc ldsb instruction which sign extends the byte in one go, versus doing an unsigned load then sign extending it separately. Another task for a simple peephole optimizer....

Also fixed the unknown unary match op problem - sign extension code was unfinished. genSwitch here we come.

Wednesday, January 14, 2009

Liveness lies

Yesterday I fixed the linear allocator to handle floating point register twinning, or at least I thought I did. The output code looked ok, but the programs I tried still crashed. I ended up spending the rest of the day writing a tool (mayet) to compare the -fasm and -fvia-c versions. Mayet takes the two .s files and splits them up into parts belonging to the individual closures. It then slowly substitutes the dubious -fasm sections for the known good -fvia-c sections.

Last night I got enough of it working to find a bad -fasm closure in cg034, which tests out floating point math. Curiously, the closure itself didn't do any float or double math. This morning I hand adjusted the -fvia-c version to look like the -fasm one until it exhibited the same problem.

After some wibbling around found this:

             ld [%l1+12],%vI_s1GO
# born: %vI_s1GO

cmp %vI_s1GO,0
# r_dying: %vI_s1GO <--------- LIES

bne .Lc2cm

......
......
c2cm:
ld [%l1+8],%vI_n2cH
# born: %vI_n2cH

st %vI_n2cH,[%i0-12]
# r_dying: %vI_n2cH

sethi %hi(base_GHCziFloat_a_closure),%l2

or %l2,%lo(base_GHCziFloat_a_closure),%l2

or %g0,%vI_s1GO,%l3
# r_dying: %vI_s1GO <---------


This is a dump of register liveness information. The line marked LIES shows that the allocator thinks that variable %vI_s1G0 isn't used after the cmp instruction. Unfortuntately, after the branch, it's used in an or. The vreg %vI_n2cH got allocated to the same register as %vI_s1G0, clobbering the contained value and causing the crash.

Turns out the register liveness determinator wasn't treating BI and BF as though they were branch instructions, so liveness information wasn't being propagated across the basic blocks properly.

Fixing that problem stopped cg034 from crashing, though it still gave the wrong answer. During debugging, noticed that if ghc is executed with -v or -ddump-reg-liveness then the top level labels emitted in the .s file change - which confuses mayet. Hmm.. let that be a lesson to all of us: changing compiler flags should not change top level names, if at all possible.

More digging

        (_s1Ri::F32,) = foreign "ccall" 
__encodeFloat((_c2sm::I32, `signed'), (_c2sn::I32, PtrHint),
(_c2so::I32, `signed'))[_unsafe_call_];
F32[Sp] = _s1Ri::F32;

Is translated to:
        call __int_encodeFloat,2
nop

st %f28,[%i0] <- BOGUS %f28

A floating point return value should be placed in %f0, but for some reason the GHC code that does just that was missing. Fixed that, and it almost works... just gives the wrong answer.

Loading of doubles looks broken.

via-c says:

ld [%l1+3], %f8
fitod %f8, %f2


but the NGC does:

ld [%l1+3],%l0
st %l0,[%o6-8]
ld [%o6-8],%f10
fitos %f10,%f10

Hmm.

Remember that comment from a few days ago:
-- ToDo: Verify correctness

Turns out it wasn't correct.. Who would have known :P

That fixed cg034 and cg035. Now we're down to:

Unexpected failures:
2080(optasm) -- segv
cg015(optasm) -- unknown unary match op
cg021(optasm) -- segv
cg022(optasm) -- segv
cg026(optasm) -- segv
cg044(optasm) -- segv
cg046(optasm) -- segv
cg054(optasm) -- genSwitch
cg058(optasm) -- segv
cg060(optasm) -- segv

Monday, January 12, 2009

Bootstrapping 7

I'm still chasing down the source of that bug from last time. Spent some time re-reading the STG paper, and trolling through the RTS code.

Running the program with +RTS -DS didn't help, but I noticed a flag -Da that checks the format of closures while the program runs. However, it seemed to be disabled, or unused. There is code for it is in the RTS, but it doesn't show up in +RTS --help. Running the program with +RTS -Da gives:

benl@mavericks:~/devel/ghc/ghc-HEAD-native/tmp> ./Main +RTS -DS -Da
stg_ap_v_ret... PAP/1(3f5212, 3ef238)
stg_ap_0_ret... Bus error (core dumped)

Went chasing through the RTS code looking for the source of the Bus Error. -Da causes Printer.c:printClosure to be invoked print out a description of each closure, but the pointers passed to it are misaligned. That is, misaligned / containing pointer tag bits. Reread the dynamic pointer tagging paper, then fixed the RTS code.

Ended up doing a binary-ish search to find the problem. Starting with a known good .s file generated with -fvia-c, slowly copied dubious sections of the -fasm version into it, testing along the way. This works because in the current STG -> Cmm translation, top level STG functions only share data via pinned registers. This is sort of like a standard calling convention for GHC functions, so it doesn't matter if GHC and GCC do register allocation a little differently. It might be worthwhile automating this process if I hit similar problems in the future.

Anyway, it turn's out there's a world of difference between
mov %l1, %l2 and mov %l2, %l1...

With that fixed, ran the should_run codeGen tests and got

OVERALL SUMMARY for test run started at Monday, 12 January 2009  5:23:10 PM EST
61 total tests, which gave rise to
427 test cases, of which
0 caused framework failures
367 were skipped

42 expected passes
1 expected failures
0 unexpected passes
17 unexpected failures

Unexpected failures:
1852(optasm) -- regalloc
1861(optasm) -- regalloc
2080(optasm) -- wrong output
cg015(optasm) -- unknown unary match op
cg018(optasm) -- regalloc
cg021(optasm) -- segv
cg022(optasm) -- segv
cg024(optasm) -- regalloc
cg026(optasm) -- regalloc
cg028(optasm) -- regalloc
cg034(optasm) -- regalloc
cg035(optasm) -- regalloc
cg044(optasm) -- regalloc
cg046(optasm) -- segv
cg054(optasm) -- genSwitch not implemented
cg058(optasm) -- segv
cg060(optasm) -- segv


The ones marked spill die with:
ghc: panic! (the 'impossible' happened)
(GHC version 6.11.20090110 for sparc-sun-solaris2):
RegAllocLinear.allocRegsAndSpill: no spill candidates


Repaired the rot in the linear register allocator. The free register map only worked for x86(_64) and PPC. Now we've got:

   2080(optasm)    -- wrong output
cg015(optasm) -- unknown unary match op
cg021(optasm) -- segv
cg022(optasm) -- segv
cg026(optasm) -- segv
cg034(optasm) -- regalloc FF64
cg035(optasm) -- regalloc FF64
cg044(optasm) -- regalloc FF64
cg046(optasm) -- segv
cg054(optasm) -- genSwitch
cg058(optasm) -- segv
cg060(optasm) -- segv


The others marked regalloc are dying because the allocator is messing up the float register twinning. That'll be tomorrow's problem.

Saturday, January 10, 2009

Bootstrapping 6

The build with the SPARC native code generator went through. Tried to run the test suite. For some reason when the compiler panics, the test framework runs off and tries to consume all available memory - like a fork bomb. Not sure why, maybe it was built wrong. Ended up just running individual tests by hand.

Ah, the joy of debugging assembly code. Fixed one problem where bad instructions were being generated. If you have LD [%f26 + ... ], %l1 then that's easy to find, because you can't use float regs for addressing.

Found a program from the test suite that seg faulted and spent some time reducing it to a smaller test case.
main = print (dude (1, 2))
dude xx
= case xx of
(x, y) -> y
Seems to work, but
main = sum2 [1,2]
sum2 xx
= case xx of
[] -> 0
(x:xs) -> x + sum2 xs
always returns 0, no matter what size the list being summed is.

Spent about an hour staring at the STG, Cmm and NCG code, but couldn't see anything obviously wrong. Noticed that the current NCG doesn't use branch delay slots, there's a comment in the code saying that doing so would confuse the register allocator. This'll be a first port of call for optimization once the NCG is working again.

On the other hand, to compile the sum2 program above with -O0, GCC used 420 instructions but the NCG used only 361 (doesn't actually work though).

I'm finding it hugely useful to have the via-c path still in place. Besides the obvious fact that I can still compile the libraries with it, it's comforting to know that the Cmm code is good, and I can just concentrate on the Cmm -> Asm pass. All the tricky dynamic pointer tagging and whatnot is expressed in Cmm, which makes life a lot easier for me. It's also good to have multiple -O flags, because it's an easy way to create wibbles in the Cmm code when diagnosing bugs.

Got bored staring at assembly code, and went back to find a better test case. Ended up with:
main = print (up 0)
up x
= case x of
0 -> x


Note that up also takes a Num dictionary. Got:
Main: internal error: stg_ap_pp_ret
(GHC version 6.11.20090105 for sparc_sun_solaris2)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
Abort (core dumped)

Bingo! Assertions in the RTS code are my new best friends.

Wednesday, January 7, 2009

Bootstrapping 5

Re-enabled compilation of the native code generator for SPARC, then went through and plugged enough holes to get it to compile again. It looks like the representation of operand formats was changed in the recent Cmm work, but the SPARC code didn't keep up because it was hidden inside #ifdef blocks. Remember kids, #ifdef = evil. Besides operand formats and the missing genSwitch function, it doesn't look like the SPARC backend has rotted too badly.

All the NCG modules have recompiled, but I must have touched something at the root of the module dependency tree. It's recompiling TcRnDriver.lhs again, which has a 2MB intermediate .hc file and has been going for 40 mins.. sigh.

Spent some time comparing the .s files produced by GCC and the GCC for Sun Systems (gccfss) compiler. It seems that any .hs file compiled via gccfss generates bad .s code because it doesn't handle pinning of STG registers the way real GCC does.

Tried to speed up recompilation by slurping across the object and interface files for TcRnDriver from a previous build. For some reason it accepted TcRnDriver.hi but not Parser.hi

parser/HaddockLex.x:23:0:
Bad interface file: dist-stage1/build/Parser.hi
Something is amiss; requested module ghc-6.11.20090105:Parser differs from
name found in the interface file ghc-6.11.20081231:Parser

Curses @ sensible checking of interface file versions! :) It appears as though the configure date (or latest patch date?) is part of the GHC version, so the backup build tree I squirreled away isn't going to help me. I'll have to leave it building overnight and remember not to run ./configure again.

In other news, a stage3 build of the HEAD + yesterdays patch worked, so at least the via-c path is good again (modulo epic slowness).

Tuesday, January 6, 2009

Bootstrapping 4

Test of patched GHC 6.10.1 produced the following:

Unexpected failures:

1861(normal,optc,hpc,threaded1,threaded2)
1914(ghci)
2594(normal,optc,hpc,threaded1,threaded2)
andy_cherry(hpc)
arith011(optc,hpc,threaded2)
barton-mangler-bug(normal,optc,hpc,threaded1,threaded2)
break024(ghci)
cabal01(normal)
conc042(threaded2)
conc043(threaded2)
conc044(threaded2)
conc045(threaded2)
concprog001(hpc)
driver062.1(normal)
driver062.2(normal)
driver062.3(normal)
driver062.4(normal)
driver062.5(normal)
driver081.1(normal)
driver081.2(normal)
dynamic_flags_001(normal)
enum01(optc,hpc,threaded2)
enum02(optc,hpc,threaded2)
enum03(optc,hpc,threaded2)
ffi019(normal,optc,hpc,ghci,threaded1,threaded2)
gadt23(normal)
ghciprog004(normal)
hClose002(normal,optc,hpc,ghci,threaded1,threaded2)
hpc_raytrace(normal,optc,hpc,threaded1,threaded2)
joao-circular(normal,optc,hpc,threaded1,threaded2)
num012(normal,optc,hpc,ghci,threaded1,threaded2)
process007(normal,optc,hpc,ghci,threaded1,threaded2)
random1283(threaded2)
recomp001(normal)
recomp004(normal)
seward-space-leak(ghci)
tcrun007(normal,optc,hpc,ghci,threaded1,threaded2)
testblockalloc(normal,threaded1)
user001(normal,optc,hpc,ghci,threaded1,threaded2)


When building the ghc-HEAD-native with the native code generator turned on, looks like the genSwitch function for SPARC is missing from MachCodeGen.hs. Also missing are the ALLOCATABLE_REGS defs from MachRegs.lhs.

Went through the description of the register set in includes/MachRegs.h. Looks like there are 3 allocatable integer regs, 6 allocatable double regs, and 4 allocatable float regs. The graph coloring allocator currently only allocates doubles, not single precision floats as well. Will have to come back and check this.

isRegRegMove and mkRegRegMovInstr from RegAllocInfo.hs were missing. So was regDotColor from RegAllocStats.hs, isFloatSize from MachRegs.lhs

MachRegs.lhs has a function mkVReg which determines the vreg to use for a particular size word. This is the source of single precision floating point vregs. All the word size code has also rotted. The sparc Size type is different from the ones use for the i386(64) and ppc. Should come back and fix this, but for now I've just added the missing functions and kept the old size type.

In MachInstrs.hs i386 has OpReg but sparc has RI, similar. Should refactor sparc code to use OpReg.

Monday, January 5, 2009

Bootstrapping 3

Running tests on patched stage2 build of GHC 6.10.1 on sparky. Most seem to be going through, but have:

cc004.hs:16:0:
calling convention not supported on this architecture: stdcall
When checking declaration:
foreign import stdcall safe "static &p" m_stdcall
:: StablePtr a -> IO (StablePtr b)

Started teasing out the SPARC stuff from nativeGen/MachCodeGen.hs. At the moment it's a mess of #ifdefery. I'm sure #ifdefs were the best way to do it back then there were only one or two targets, but now there is code for i386, i386_64, powerpc, alpha and sparc all mixed in together.

My plan is to copy out the non-architecture-specific functions from nativeGen/MachCodeGen.hs into their own module nativeGen/MachCodeGenShared.hs. I'll split the sparc specific stuff into a set of modules under nativeGen/sparc. Once the sparc native gen works again I'll then go back and delete the sparc specific stuff from nativeGen/MachCodeGen.hs, and make it use the MachCodeGenShared.hs. This should leave the original support for all architectures untouched during development.

While going through the code, found an amusing comment in the sparc section:
-- Floating point assignment to a register/temporary
-- ToDo: Verify correctness
assignReg_FltCode :: Size -> CmmReg -> CmmExpr -> NatM InstrBlock
assignReg_FltCode pk reg src = do ...

hmmm. I wonder how long that ToDo has been there..

The build of GCC 4.3.2 died with
Configuring stage 1 in sparc-sun-solaris2.10/libgcc
...
checking for suffix of object files... configure: error: cannot compute
suffix of object files: cannot compile.

Investagation of the config.log reveals:
/home/benl/files/gcc/build/gcc-4.3.2-obj/./gcc/xgcc ...
conftest.c:1: internal compiler error: Segmentation Fault
Please submit a full bug report,


Tried to back off to GCC 4.2.1 then GCC 4.2.4. Both die during the build with configure problems:
config.status: executing gstdint.h commands
make[3]: Entering directory `/home/benl/files/gcc/build/gcc-4.2.4-obj/libdecnumber'
make[3]: Nothing to be done for `all'.
make[3]: Leaving directory `/home/benl/files/gcc/build/gcc-4.2.4-obj/libdecnumber'
make[3]: Entering directory `/home/benl/files/gcc/build/gcc-4.2.4-obj/gcc'
make[3]: *** No rule to make target `all'. Stop.
make[3]: Leaving directory `/home/benl/files/gcc/build/gcc-4.2.4-obj/gcc'

For some reason the configure script isn't dropping the Makefile in ./gcc.

Giving up trying to compile a more recent GCC under Solaris. Will just copy the 4.2.1 binaries across from mavericks. Let's hope we don't stumble across any more bugs in it.

Stop. Rewind. I've been tripping over myself because the builds are taking so long. I've got copies of the head on mavericks, code.haskell.org and sparky and they're all different versions. Some of the recent patches pushed to the head also seem to have broken the build, so I'm backing up to the head as of 2009/01/01.

I'm going to stick with this version, and the same compiler flags, for all builds until the NGC is fixed. I need to be able to reuse the .o files between builds because they take too long to remake.

Finally pushed the gc_thread patch. This should fix the via-c build. I've run through the testsuite, though I'm still waiting the actual stage2 build to finish - it's stuck on Parser.hs again.

Turned on the NCG for sparc. That'll be another nights worth of rebuilding.