Reword study info.
[software/elephly-net.git] / 2017-01-09-bootstrapping-haskell-part-1.org
1 #+TITLE: Bootstrapping Haskell: part 1
2 #+AUTHOR: Ricardo Wurmus
3 #+DATE: <2017-01-08 Sun>
4
5 Haskell is a formally specified language with potentially many alternative implementations, but in early 2017 the reality is that Haskell is whatever the Glasgow Haskell Compiler (GHC) implements. Unfortunately, to build GHC one needs a previous version of GHC. This is true for all public releases of GHC all the way back to version 0.29, which was released in 1996 and which implements Haskell 1.2. Some GHC releases include files containing generated ANSI C code, which require only a C compiler to build. For most purposes, generated code does not qualify as source code.
6
7 So I wondered: /is it possible to construct a procedure to build a modern release of GHC from source without depending on any generated code or pre-built binaries of an older variant of GHC?/ The answer to this question depends on the answers to a number of related questions. One of them is: are there any alternative Haskell implementations that are still usable today and that can be built without GHC?
8
9 * A short survey of Haskell implementations
10
11 Although nowadays hardly anyone uses any other Haskell compiler but GHC in production there are some alternative Haskell implementations that were protected from bit rot and thus can still be built from source with today’s common toolchains.
12
13 One of the oldest implementations is [[http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/syntax/haskell/0.html][Yale Haskell]], a Haskell system embedded in Common Lisp. The last release of Yale Haskell was version 2.0.5 in the early 1990s.[fn::It is unclear when exactly the release was made, but any time between 1991 and 1993 seems likely.] Yale Haskell runs on top of CMU Common Lisp, Lucid Common Lisp, Allegro Common Lisp, or Harlequin LispWorks, but since I do not have access to any of these proprietary Common Lisp implementations, I ported the Yale Haskell system to GNU CLISP. The [[http://git.elephly.net/software/yale-haskell.git][code for the port is available here]]. Yale Haskell is not a compiler, it can only be used as an interpreter.
14
15 Another Haskell interpreter with a more recent release is [[https://haskell.org/hugs][Hugs]]. Hugs is written in C and implements almost all of the Haskell 98 standard. It also comes with a number of [[https://www.haskell.org/hugs/pages/users_guide/hugs-ghc.htm][useful language extensions]] that GHC and other Haskell systems depend on. Unfortunately, it cannot deal with mutually recursive module dependencies, which is a feature that even the earliest versions of GHC rely on. This means that running a variant of GHC inside of Hugs is not going to work without major changes.
16
17 An alternative Haskell compiler that does not need to be built with GHC is [[https://www.haskell.org/nhc98][nhc98]]. Its latest release was in 2010, which is much more recent than any of the other Haskell implementations mentioned so far. nhc98 is written in Haskell, so a Haskell compiler or interpreter is required to build it. Like GHC the release of nhc98 comes with files containing generated C code, but depending on them for a clean bootstrap is almost as bad as depending on a third-party binary. Sadly, nhc98 has another shortcoming: it is restricted to 32-bit machine architectures.
18
19 * An early bootstrapping idea
20
21 Since nhc98 is written in C (the runtime) and standard Haskell 98, we can run the Haskell parts of the compiler inside of a Haskell 98 interpreter. Luckily, we have an interpreter that fits the bill: Hugs! If we can interpret and run enough parts of nhc98 with Hugs we might be able to use nhc98 on Hugs to build a native version of nhc98 and related tools (such as cpphs, hmake, and cabal). Using the native compiler we can build a complete toolchain and just /maybe/ that’s enough to build an early version of GHC. Once we have an early version of GHC we can go ahead and build later versions with ease. (Building GHC directly using nhc98 on Hugs might also work, but due to complexity in GHC modules it seems better to avoid depending on Hugs at runtime.)
22
23 At this point I have verified that (with minor modifications) nhc98 can indeed be run on top of Hugs and that (with enough care to pre-processing and dependency ordering) it can build a native library from the Haskell source files of the nhc98 prelude. It is not clear whether nhc98 would be capable of building a version of GHC and how close to a modern version GHC we can get with just nhc98. There are also problems with the nhc98 runtime on modern x86_64 systems (more on that at the end).
24
25 * Setting up the environment
26
27 Before we can start let’s prepare a suitable environment. Since nhc98 can only be used on 32-bit architectures we need a GCC toolchain for i686. With [[https://gnu.org/software/guix][GNU Guix]] it’s easy to set up a temporary environment containing just the right tools: the GCC toolchain, make, and Hugs.
28
29 #+BEGIN_SRC sh
30 guix environment --system=i686-linux \
31 --ad-hoc gcc-toolchain@4 make hugs
32 #+END_SRC
33
34 * Building the runtime
35
36 Now we can configure nhc98 and build the C runtime, which is needed to link the binary objects that nhc98 and GCC produce when compiling Haskell sources. Configuration is easy:
37
38 #+BEGIN_SRC sh
39 cd /path/to/nhc98-1.22
40 export NHCDIR=$PWD
41 ./configure
42 #+END_SRC
43
44 Next, we build the C runtime:
45
46 #+BEGIN_SRC sh
47 cd src/runtime
48 make
49 #+END_SRC
50
51 This produces binary objects in an architecture-specific directory. In my case this is =targets/x86_64-Linux=.
52
53 * Building the prelude?
54
55 The standard library in Haskell is called the prelude. Most source files of nhc98 depend on the prelude in one way or another. Although Hugs comes with its own prelude it is of little use for our purposes as components of nhc98 must be linked with a static prelude library object. Hugs does not provide a suitable object that we could link to.
56
57 To build the prelude we need a Haskell compiler that has the same call interface as nhc98. Interestingly, much of the nhc98 compiler’s user interface is implemented as a shell script, which after extensive argument processing calls =nhc98comp= to translate Haskell source files into C code, and then runs GCC over the C files to create binary objects. Since we do not have =nhc98comp= at this point, we need to fake it with Hugs (more on that later).
58
59 * Detour: cpphs and GreenCard
60
61 Unfortunately, this is not the only problem. Some of the prelude’s source files require pre-processing by a tool called GreenCard, which generates boilerplate FFI code. Of course, GreenCard is written in Haskell. Since we cannot build a native GreenCard binary without the native nhc98 prelude library, we need to make GreenCard run on Hugs. Some of the GreenCard sources require pre-processing with =cpphs=. Luckily, that’s just a really simple Haskell script, so running it in Hugs is trivial. We will need =cpphs= later again, so it makes sense to write a script for it. Let’s call it =hugs-cpphs= and drop it in =${NHCDIR}=.
62
63 #+BEGIN_SRC sh
64 #!/bin/bash
65
66 runhugs ${NHCDIR}/src/cpphs/cpphs.hs --noline -D__HASKELL98__ "$@"
67 #+END_SRC
68
69 Make it executable:
70
71 #+BEGIN_SRC sh
72 chmod +x hugs-cpphs
73 #+END_SRC
74
75 Okay, let’s first pre-process the GreenCard sources with =cpphs=. To do that I ran the following commands:
76
77 #+BEGIN_SRC sh
78 cd ${NHCDIR}/src/greencard
79
80 CPPPRE="${NHCDIR}/hugs-cpphs -D__NHC__"
81
82 FILES="DIS.lhs \
83 HandLex.hs \
84 ParseLib.hs \
85 HandParse.hs \
86 FillIn.lhs \
87 Proc.lhs \
88 NHCBackend.hs \
89 NameSupply.lhs \
90 Process.lhs"
91
92 for file in $FILES; do
93 cp $file $file.original && $CPPPRE $file.original > $file && rm $file.original
94 done
95 #+END_SRC
96
97 The result is a bunch of GreenCard source files without these pesky CPP pre-processor directives. Hugs can pre-process sources on the fly, but this makes evaluation orders of magnitude slower. Pre-processing the sources once before using them repeatedly seems like a better choice.
98
99 There is still a minor problem with GreenCard on Hugs. The GreenCard sources import the module =NonStdTrace=, which depends on built-in prelude functions from nhc98. Obviously, they are not available when running on Hugs (it has its own prelude implementation), so we need to provide an alternative using just the regular Hugs prelude. The following snippet creates a file named =src/prelude/NonStd/NonStdTraceBootstrap.hs= with the necessary changes.
100
101 #+BEGIN_SRC sh
102 cd ${NHCDIR}/src/prelude/NonStd/
103 sed -e 's|NonStdTrace|NonStdTraceBootstrap|' \
104 -e 's|import PreludeBuiltin||' \
105 -e 's|_||g' NonStdTrace.hs > NonStdTraceBootstrap.hs
106 #+END_SRC
107
108 Then we change a single line in =src/greencard/NHCBackend.hs= to make it import =NonStdTraceBootstrap= instead of =NonStdTrace=.
109
110 #+BEGIN_SRC sh
111 cd ${NHCDIR}/src/greencard
112 sed -i -e 's|NonStdTrace|NonStdTraceBootstrap|' NHCBackend.hs
113 #+END_SRC
114
115 To run GreenCard we still need a driver script. Let’s call this =hugs-greencard= and place it in =${NHCDIR}=:
116
117 #+BEGIN_SRC sh
118 #!/bin/bash
119
120 HUGSDIR="$(dirname $(readlink -f $(which runhugs)))/../"
121 SEARCH_HUGS=$(printf "${NHCDIR}/src/%s/*:" compiler prelude libraries)
122
123 runhugs -98 \
124 -P${HUGSDIR}/lib/hugs/packages/*:${NHCDIR}/include/*:${SEARCH_HUGS} \
125 ${NHCDIR}/src/greencard/GreenCard.lhs \
126 $@
127 #+END_SRC
128
129 Make it executable:
130
131 #+BEGIN_SRC sh
132 cd ${NHCDIR}
133 chmod +x hugs-greencard
134 #+END_SRC
135
136 * Building the prelude!
137
138 Where were we? Ah, the prelude. As stated earlier, we need a working replacement for =nhc98comp=, which will be called by the driver script =script/nhc98= (created by the configure script). Let’s call the replacement =hugs-nhc=, and again we’ll dump it in =${NHCDIR}=. Here it is in all its glory:
139
140 #+BEGIN_SRC sh
141 #!/bin/bash
142
143 # Root directory of Hugs installation
144 HUGSDIR="$(dirname $(readlink -f $(which runhugs)))/../"
145
146 # TODO: "libraries" alone may be sufficient
147 SEARCH_HUGS=$(printf "${NHCDIR}/src/%s/*:" compiler prelude libraries)
148
149 # Filter everything from "+RTS" to "-RTS" from $@ because MainNhc98.hs
150 # does not know what to do with these flags.
151 ARGS=""
152 SKIP="false"
153 for arg in "$@"; do
154 if [[ $arg == "+RTS" ]]; then
155 SKIP="true"
156 elif [[ $arg == "-RTS" ]]; then
157 SKIP="false"
158 elif [[ $SKIP == "false" ]]; then
159 ARGS="${ARGS} $arg"
160 fi
161 done
162
163 runhugs -98 \
164 -P${HUGSDIR}/lib/hugs/packages/*:${SEARCH_HUGS} \
165 ${NHCDIR}/src/compiler98/MainNhc98.hs \
166 $ARGS
167 #+END_SRC
168
169 All this does is run Hugs (=runhugs=) with language extensions (=-98=), ensures that Hugs knows where to look for Hugs and nhc98 modules (=-P=), loads up the compiler’s =main= function, and then passes any arguments other than RTS flags (=$ARGS=) to it.
170
171 Let’s also make this executable:
172
173 #+BEGIN_SRC sh
174 cd ${NHCDIR}
175 chmod +x hugs-nhc
176 #+END_SRC
177
178 The compiler sources contain pre-processor directives, which need to be removed before running =hugs-nhc=. It would be foolish to let Hugs pre-process the sources at runtime with =-F=. In my tests it made =hugs-nhc= run slower by an order of magnitude. Let’s pre-process the sources of the compiler and the libraries it depends on with =hugs-cpphs= (see above):
179
180 #+BEGIN_SRC sh
181 cd ${NHCDIR}
182 CPPPRE="${NHCDIR}/hugs-cpphs -D__HUGS__"
183
184 FILES="src/compiler98/GcodeLowC.hs \
185 src/libraries/filepath/System/FilePath.hs \
186 src/libraries/filepath/System/FilePath/Posix.hs"
187
188 for file in $FILES; do
189 cp $file $file.original && $CPPPRE $file.original > $file && rm $file.original
190 done
191 #+END_SRC
192
193 The compiler’s driver script =script/nhc98= expects to find the executables of =hmake-PRAGMA=, =greencard-nhc98=, and =cpphs= in the architecture-specific lib directory (in my case that’s =${NHCDIR}/lib/x86_64-Linux/=). They do not exist, obviously, but for two of them we already have scripts to run them on top of Hugs. =hmake-PRAGMA= does not seem to be very important; replacing it with =cat= appears to be fine. To pacify the compiler script it’s easiest to just replace a few definitions:
194
195 #+BEGIN_SRC sh
196 cd ${NHCDIR}
197 sed -i \
198 -e '0,/^GREENCARD=.*$/s||GREENCARD="$NHC98BINDIR/../hugs-greencard"|' \
199 -e '0,/^CPPHS=.*$/s||CPPHS="$NHC98BINDIR/../hugs-cpphs -D__NHC__"|' \
200 -e '0,/^PRAGMA=.*$/s||PRAGMA=cat|' \
201 script/nhc98
202 #+END_SRC
203
204 Initially, this looked like it would be enough, but half-way through building the prelude Hugs choked when interpreting nhc98 to build a certain module. After some experimentation it turned out that the =NHC.FFI= module in =src/prelude/FFI/CTypes.hs= is too big for Hugs. Running nhc98 on that module causes Hugs to abort with an overflow in the control stack. The fix here is to break up the module to make it easier for nhc98 to build it, which in turn prevents Hugs from doing too much work at once.
205
206 Apply this patch:
207
208 #+BEGIN_EXAMPLE
209 From 9eb2a2066eb9f93e60e447aab28479af6c8b9759 Mon Sep 17 00:00:00 2001
210 From: Ricardo Wurmus <rekado@elephly.net>
211 Date: Sat, 7 Jan 2017 22:31:41 +0100
212 Subject: [PATCH] Split up CTypes
213
214 This is necessary to avoid a control stack overflow in Hugs when
215 building the FFI library with nhc98 running on Hugs.
216 ---
217 src/prelude/FFI/CStrings.hs | 2 ++
218 src/prelude/FFI/CTypes.hs | 14 --------------
219 src/prelude/FFI/CTypes1.hs | 20 ++++++++++++++++++++
220 src/prelude/FFI/CTypes2.hs | 22 ++++++++++++++++++++++
221 src/prelude/FFI/CTypesExtra.hs | 2 ++
222 src/prelude/FFI/FFI.hs | 2 ++
223 src/prelude/FFI/Makefile | 8 ++++----
224 src/prelude/FFI/MarshalAlloc.hs | 2 ++
225 src/prelude/FFI/MarshalUtils.hs | 2 ++
226 9 files changed, 56 insertions(+), 18 deletions(-)
227 create mode 100644 src/prelude/FFI/CTypes1.hs
228 create mode 100644 src/prelude/FFI/CTypes2.hs
229
230 diff --git a/src/prelude/FFI/CStrings.hs b/src/prelude/FFI/CStrings.hs
231 index 18fdfa9..f1373cf 100644
232 --- a/src/prelude/FFI/CStrings.hs
233 +++ b/src/prelude/FFI/CStrings.hs
234 @@ -23,6 +23,8 @@ module NHC.FFI (
235
236 import MarshalArray
237 import CTypes
238 +import CTypes1
239 +import CTypes2
240 import Ptr
241 import Word
242 import Char
243 diff --git a/src/prelude/FFI/CTypes.hs b/src/prelude/FFI/CTypes.hs
244 index 18e9d60..942e7a1 100644
245 --- a/src/prelude/FFI/CTypes.hs
246 +++ b/src/prelude/FFI/CTypes.hs
247 @@ -4,11 +4,6 @@ module NHC.FFI
248 -- Typeable, Storable, Bounded, Real, Integral, Bits
249 CChar(..), CSChar(..), CUChar(..)
250 , CShort(..), CUShort(..), CInt(..), CUInt(..)
251 - , CLong(..), CULong(..), CLLong(..), CULLong(..)
252 -
253 - -- Floating types, instances of: Eq, Ord, Num, Read, Show, Enum,
254 - -- Typeable, Storable, Real, Fractional, Floating, RealFrac, RealFloat
255 - , CFloat(..), CDouble(..), CLDouble(..)
256 ) where
257
258 import NonStdUnsafeCoerce
259 @@ -29,12 +24,3 @@ INTEGRAL_TYPE(CShort,Int16)
260 INTEGRAL_TYPE(CUShort,Word16)
261 INTEGRAL_TYPE(CInt,Int)
262 INTEGRAL_TYPE(CUInt,Word32)
263 -INTEGRAL_TYPE(CLong,Int32)
264 -INTEGRAL_TYPE(CULong,Word32)
265 -INTEGRAL_TYPE(CLLong,Int64)
266 -INTEGRAL_TYPE(CULLong,Word64)
267 -
268 -FLOATING_TYPE(CFloat,Float)
269 -FLOATING_TYPE(CDouble,Double)
270 --- HACK: Currently no long double in the FFI, so we simply re-use double
271 -FLOATING_TYPE(CLDouble,Double)
272 diff --git a/src/prelude/FFI/CTypes1.hs b/src/prelude/FFI/CTypes1.hs
273 new file mode 100644
274 index 0000000..81ba0f5
275 --- /dev/null
276 +++ b/src/prelude/FFI/CTypes1.hs
277 @@ -0,0 +1,20 @@
278 +{-# OPTIONS_COMPILE -cpp #-}
279 +module NHC.FFI
280 + ( CLong(..), CULong(..), CLLong(..), CULLong(..)
281 + ) where
282 +
283 +import NonStdUnsafeCoerce
284 +import Int ( Int8, Int16, Int32, Int64 )
285 +import Word ( Word8, Word16, Word32, Word64 )
286 +import Storable ( Storable(..) )
287 +-- import Data.Bits( Bits(..) )
288 +-- import NHC.SizedTypes
289 +import Monad ( liftM )
290 +import Ptr ( castPtr )
291 +
292 +#include "CTypes.h"
293 +
294 +INTEGRAL_TYPE(CLong,Int32)
295 +INTEGRAL_TYPE(CULong,Word32)
296 +INTEGRAL_TYPE(CLLong,Int64)
297 +INTEGRAL_TYPE(CULLong,Word64)
298 diff --git a/src/prelude/FFI/CTypes2.hs b/src/prelude/FFI/CTypes2.hs
299 new file mode 100644
300 index 0000000..7d66242
301 --- /dev/null
302 +++ b/src/prelude/FFI/CTypes2.hs
303 @@ -0,0 +1,22 @@
304 +{-# OPTIONS_COMPILE -cpp #-}
305 +module NHC.FFI
306 + ( -- Floating types, instances of: Eq, Ord, Num, Read, Show, Enum,
307 + -- Typeable, Storable, Real, Fractional, Floating, RealFrac, RealFloat
308 + CFloat(..), CDouble(..), CLDouble(..)
309 + ) where
310 +
311 +import NonStdUnsafeCoerce
312 +import Int ( Int8, Int16, Int32, Int64 )
313 +import Word ( Word8, Word16, Word32, Word64 )
314 +import Storable ( Storable(..) )
315 +-- import Data.Bits( Bits(..) )
316 +-- import NHC.SizedTypes
317 +import Monad ( liftM )
318 +import Ptr ( castPtr )
319 +
320 +#include "CTypes.h"
321 +
322 +FLOATING_TYPE(CFloat,Float)
323 +FLOATING_TYPE(CDouble,Double)
324 +-- HACK: Currently no long double in the FFI, so we simply re-use double
325 +FLOATING_TYPE(CLDouble,Double)
326 diff --git a/src/prelude/FFI/CTypesExtra.hs b/src/prelude/FFI/CTypesExtra.hs
327 index ba3f15b..7cbdcbb 100644
328 --- a/src/prelude/FFI/CTypesExtra.hs
329 +++ b/src/prelude/FFI/CTypesExtra.hs
330 @@ -20,6 +20,8 @@ import Storable ( Storable(..) )
331 import Monad ( liftM )
332 import Ptr ( castPtr )
333 import CTypes
334 +import CTypes1
335 +import CTypes2
336
337 #include "CTypes.h"
338
339 diff --git a/src/prelude/FFI/FFI.hs b/src/prelude/FFI/FFI.hs
340 index 9d91e57..0c29394 100644
341 --- a/src/prelude/FFI/FFI.hs
342 +++ b/src/prelude/FFI/FFI.hs
343 @@ -217,6 +217,8 @@ import MarshalUtils -- routines for basic marshalling
344 import MarshalError -- routines for basic error-handling
345
346 import CTypes -- newtypes for various C basic types
347 +import CTypes1
348 +import CTypes2
349 import CTypesExtra -- types for various extra C types
350 import CStrings -- C pointer to array of char
351 import CString -- nhc98-only
352 diff --git a/src/prelude/FFI/Makefile b/src/prelude/FFI/Makefile
353 index 99065f8..e229672 100644
354 --- a/src/prelude/FFI/Makefile
355 +++ b/src/prelude/FFI/Makefile
356 @@ -18,7 +18,7 @@ EXTRA_C_FLAGS =
357 SRCS = \
358 Addr.hs Ptr.hs FunPtr.hs Storable.hs \
359 ForeignObj.hs ForeignPtr.hs Int.hs Word.hs \
360 - CError.hs CTypes.hs CTypesExtra.hs CStrings.hs \
361 + CError.hs CTypes.hs CTypes1.hs CTypes2.hs CTypesExtra.hs CStrings.hs \
362 MarshalAlloc.hs MarshalArray.hs MarshalError.hs MarshalUtils.hs \
363 StablePtr.hs
364
365 @@ -38,12 +38,12 @@ Word.hs: Word.hs.cpp
366 # dependencies generated by hmake -Md: (and hacked by MW)
367 ${OBJDIR}/MarshalError.$O: ${OBJDIR}/Ptr.$O
368 ${OBJDIR}/MarshalUtils.$O: ${OBJDIR}/Ptr.$O ${OBJDIR}/Storable.$O \
369 - ${OBJDIR}/MarshalAlloc.$O ${OBJDIR}/CTypes.$O ${OBJDIR}/CTypesExtra.$O
370 + ${OBJDIR}/MarshalAlloc.$O ${OBJDIR}/CTypes.$O ${OBJDIR}/CTypes1.$O ${OBJDIR}/CTypes2.$O ${OBJDIR}/CTypesExtra.$O
371 ${OBJDIR}/MarshalArray.$O: ${OBJDIR}/Ptr.$O ${OBJDIR}/Storable.$O \
372 ${OBJDIR}/MarshalAlloc.$O ${OBJDIR}/MarshalUtils.$O
373 -${OBJDIR}/CTypesExtra.$O: ${OBJDIR}/Int.$O ${OBJDIR}/Word.$O ${OBJDIR}/CTypes.$O
374 +${OBJDIR}/CTypesExtra.$O: ${OBJDIR}/Int.$O ${OBJDIR}/Word.$O ${OBJDIR}/CTypes.$O ${OBJDIR}/CTypes1.$O ${OBJDIR}/CTypes2.$O
375 ${OBJDIR}/CTypes.$O: ${OBJDIR}/Int.$O ${OBJDIR}/Word.$O ${OBJDIR}/Storable.$O \
376 - ${OBJDIR}/Ptr.$O
377 + ${OBJDIR}/Ptr.$O ${OBJDIR}/CTypes1.$O ${OBJDIR}/CTypes2.$O
378 ${OBJDIR}/CStrings.$O: ${OBJDIR}/MarshalArray.$O ${OBJDIR}/CTypes.$O \
379 ${OBJDIR}/Ptr.$O ${OBJDIR}/Word.$O
380 ${OBJDIR}/MarshalAlloc.$O: ${OBJDIR}/Ptr.$O ${OBJDIR}/Storable.$O \
381 diff --git a/src/prelude/FFI/MarshalAlloc.hs b/src/prelude/FFI/MarshalAlloc.hs
382 index 34ac7b3..5b43554 100644
383 --- a/src/prelude/FFI/MarshalAlloc.hs
384 +++ b/src/prelude/FFI/MarshalAlloc.hs
385 @@ -14,6 +14,8 @@ import ForeignPtr (FinalizerPtr(..))
386 import Storable
387 import CError
388 import CTypes
389 +import CTypes1
390 +import CTypes2
391 import CTypesExtra (CSize)
392 import NHC.DErrNo
393
394 diff --git a/src/prelude/FFI/MarshalUtils.hs b/src/prelude/FFI/MarshalUtils.hs
395 index 312719b..bd9d149 100644
396 --- a/src/prelude/FFI/MarshalUtils.hs
397 +++ b/src/prelude/FFI/MarshalUtils.hs
398 @@ -29,6 +29,8 @@ import Ptr
399 import Storable
400 import MarshalAlloc
401 import CTypes
402 +import CTypes1
403 +import CTypes2
404 import CTypesExtra
405
406 -- combined allocation and marshalling
407 --
408 2.11.0
409 #+END_EXAMPLE
410
411 After all this it’s time for a break. Run the following commands for a long break:
412
413 #+BEGIN_SRC sh
414 cd ${NHCDIR}/src/prelude
415 time make NHC98COMP=$NHCDIR/hugs-nhc
416 #+END_SRC
417
418 After the break—it took more than two hours on my laptop—you should see output like this:
419
420 #+BEGIN_EXAMPLE
421 ranlib /path/to/nhc98-1.22/lib/x86_64-Linux/Prelude.a
422 #+END_EXAMPLE
423
424 Congratulations! You now have a native nhc98 prelude library!
425
426 * Building hmake
427
428 The compiler and additional Haskell libraries all require a tool called “hmake” to automatically order dependencies, so we’ll try to build it next. There’s just a small problem with one of the source files: =src/hmake/FileName.hs= contains the name “Niklas Röjemo” and the compiler really does not like the umlaut. With apologies to Niklas we change the copyright line to appease the compiler.
429
430 #+BEGIN_SRC sh
431 cd $NHCDIR/src/hmake
432 mv FileName.hs{,.broken}
433 tr '\366' 'o' < FileName.hs.broken > FileName.hs
434 rm FileName.hs.broken
435 NHC98COMP=$NHCDIR/hugs-nhc make HC=$NHCDIR/script/nhc98
436 #+END_SRC
437
438 * To be continued
439
440 Unfortunately, the hmake tools are not working. All of the tools (e.g. =MkConfig=) fail with an early segmentation fault. There must be an error in the runtime, likely in =src/runtime/Kernel/mutator.c= where bytecode for heap and stack operations is interpreted. One thing that looks like a problem is statements like this:
441
442 #+BEGIN_EXAMPLE
443 *--sp = (NodePtr) constptr[-HEAPOFFSET(ip[0])];
444 #+END_EXAMPLE
445
446 =constptr= is =NULL=, so this seems to be just pointer arithmetic expressed in array notation. These errors can be fixed by rewriting the statement to use explicit pointer arithmetic:
447
448 #+BEGIN_EXAMPLE
449 *--sp = (NodePtr) (constptr + (-HEAPOFFSET(ip[0])));
450 #+END_EXAMPLE
451
452 Unfortunately, this doesn’t seem to be enough as there is another segfault in the handling of the =EvalTOS= label. =IND_REMOVE= is applied to the contents of the stack pointer, which turns out to be =0x10=, which just doesn’t seem right. =IND_REMOVE= removes indirection by following pointer addresses until the value stored at the given address does not look like an address. This fails because =0x10= does look like an address—it’s just invalid. I have enabled a bunch of tracing and debugging features, but I don’t fully understand how the nhc98 runtime is /supposed/ to work.
453
454 Judging from mails on the nhc-bugs and nhc-users lists I see that I’m not the only one experiencing segfaults. [[https://mail.haskell.org/pipermail/nhc-bugs/2005-October/000529.html][This email]] suggests that segfaults are “associated with changes in the way gcc lays out static arrays of bytecodes, e.g. by putting extra padding space between arrays that are supposed to be adjacent.” I may have to try different compiler flags or an older version of GCC; I only tried with GCC 4.9.4 but the [[http://sources.debian.net/src/nhc98/1.16-15/debian/rules/][Debian package for nhc98]] used version 2.95 or 3.3.
455
456 For completeness sake here’s the trace of the failing execution of MkProg:
457
458 #+BEGIN_EXAMPLE
459 (gdb) run
460 Starting program: /path/to/nhc98-1.22/lib/x86_64-Linux/MkProg
461 ZAP_ARG_I1 hp=0x80c5010 sp=0x8136a10 fp=0x8136a10 ip=0x8085140
462 NEEDHEAP_I32 hp=0x80c5010 sp=0x8136a10 fp=0x8136a10 ip=0x8085141
463 HEAP_CVAL_N1 hp=0x80c5010 sp=0x8136a10 fp=0x8136a10 ip=0x8085142
464 HEAP_CVAL_I3 hp=0x80c5014 sp=0x8136a10 fp=0x8136a10 ip=0x8085144
465 HEAP_OFF_N1 hp=0x80c5018 sp=0x8136a10 fp=0x8136a10 ip=0x8085145
466 PUSH_HEAP hp=0x80c501c sp=0x8136a10 fp=0x8136a10 ip=0x8085147
467 HEAP_CVAL_I4 hp=0x80c501c sp=0x8136a0c fp=0x8136a10 ip=0x8085148
468 HEAP_CVAL_I5 hp=0x80c5020 sp=0x8136a0c fp=0x8136a10 ip=0x8085149
469 HEAP_OFF_N1 hp=0x80c5024 sp=0x8136a0c fp=0x8136a10 ip=0x808514a
470 PUSH_CVAL_P1 hp=0x80c5028 sp=0x8136a0c fp=0x8136a10 ip=0x808514c
471 PUSH_I1 hp=0x80c5028 sp=0x8136a08 fp=0x8136a10 ip=0x808514e
472 ZAP_STACK_P1 hp=0x80c5028 sp=0x8136a04 fp=0x8136a10 ip=0x808514f
473 EVAL hp=0x80c5028 sp=0x8136a04 fp=0x8136a10 ip=0x8085151
474 eval: evalToS
475
476 Program received signal SIGSEGV, Segmentation fault.
477 0x0804ac27 in run (toplevel=0x80c5008) at mutator.c:425
478 425 IND_REMOVE(nodeptr);
479 #+END_EXAMPLE
480
481 =ip= is the instruction pointer, which points at the current element in the bytecode stream. =fp= is probably the frame pointer, =sp= the stack pointer, and =hp= the heap pointer. The [[https://www.haskell.org/nhc98/implementation-notes/index.html][implementation notes for nhc98]] will probably be helpful in solving this problem.
482
483 Anyway, that’s where I’m at so far. If you are interested in these kinds of problems or other bootstrapping projects, consider joining the efforts of the [[http://bootstrappable.org][Bootstrappable Builds project]]!