2017-01-09-bootstrapping-haskell-part-1.org

   1 #+TITLE: Bootstrapping Haskell: part 1
   2 #+AUTHOR: Ricardo Wurmus
   3 #+DATE: <2017-01-08 Sun>
   4
   5 Haskell is a formally specified language with potentially many alternative implementations, but in early 2017 the reality is that Haskell is whatever the Glasgow Haskell Compiler (GHC) implements.  Unfortunately, to build GHC one needs a previous version of GHC.  This is true for all public releases of GHC all the way back to version 0.29, which was released in 1996 and which implements Haskell 1.2.  Some GHC releases include files containing generated ANSI C code, which require only a C compiler to build.  For most purposes, generated code does not qualify as source code.
   6
   7 So I wondered: /is it possible to construct a procedure to build a modern release of GHC from source without depending on any generated code or pre-built binaries of an older variant of GHC?/  The answer to this question depends on the answers to a number of related questions.  One of them is: are there any alternative Haskell implementations that are still usable today and that can be built without GHC?
   8
   9 * A short survey of Haskell implementations
  10
  11 Although nowadays hardly anyone uses any other Haskell compiler but GHC in production there are some alternative Haskell implementations that were protected from bit rot and thus can still be built from source with today’s common toolchains.
  12
  13 One of the oldest implementations is [[http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/code/syntax/haskell/0.html][Yale Haskell]], a Haskell system embedded in Common Lisp.  The last release of Yale Haskell was version 2.0.5 in the early 1990s.[fn::It is unclear when exactly the release was made, but any time between 1991 and 1993 seems likely.]  Yale Haskell runs on top of CMU Common Lisp, Lucid Common Lisp, Allegro Common Lisp, or Harlequin LispWorks, but since I do not have access to any of these proprietary Common Lisp implementations, I ported the Yale Haskell system to GNU CLISP.  The [[http://git.elephly.net/software/yale-haskell.git][code for the port is available here]].  Yale Haskell is not a compiler, it can only be used as an interpreter.
  14
  15 Another Haskell interpreter with a more recent release is [[https://haskell.org/hugs][Hugs]].  Hugs is written in C and implements almost all of the Haskell 98 standard.  It also comes with a number of [[https://www.haskell.org/hugs/pages/users_guide/hugs-ghc.htm][useful language extensions]] that GHC and other Haskell systems depend on.  Unfortunately, it cannot deal with mutually recursive module dependencies, which is a feature that even the earliest versions of GHC rely on.  This means that running a variant of GHC inside of Hugs is not going to work without major changes.
  16
  17 An alternative Haskell compiler that does not need to be built with GHC is [[https://www.haskell.org/nhc98][nhc98]].  Its latest release was in 2010, which is much more recent than any of the other Haskell implementations mentioned so far.  nhc98 is written in Haskell, so a Haskell compiler or interpreter is required to build it.  Like GHC the release of nhc98 comes with files containing generated C code, but depending on them for a clean bootstrap is almost as bad as depending on a third-party binary.  Sadly, nhc98 has another shortcoming: it is restricted to 32-bit machine architectures.
  18
  19 * An early bootstrapping idea
  20
  21 Since nhc98 is written in C (the runtime) and standard Haskell 98, we can run the Haskell parts of the compiler inside of a Haskell 98 interpreter.  Luckily, we have an interpreter that fits the bill: Hugs!  If we can interpret and run enough parts of nhc98 with Hugs we might be able to use nhc98 on Hugs to build a native version of nhc98 and related tools (such as cpphs, hmake, and cabal).  Using the native compiler we can build a complete toolchain and just /maybe/ that’s enough to build an early version of GHC.  Once we have an early version of GHC we can go ahead and build later versions with ease.  (Building GHC directly using nhc98 on Hugs might also work, but due to complexity in GHC modules it seems better to avoid depending on Hugs at runtime.)
  22
  23 At this point I have verified that (with minor modifications) nhc98 can indeed be run on top of Hugs and that (with enough care to pre-processing and dependency ordering) it can build a native library from the Haskell source files of the nhc98 prelude.  It is not clear whether nhc98 would be capable of building a version of GHC and how close to a modern version GHC we can get with just nhc98.  There are also problems with the nhc98 runtime on modern x86_64 systems (more on that at the end).
  24
  25 * Setting up the environment
  26
  27 Before we can start let’s prepare a suitable environment.  Since nhc98 can only be used on 32-bit architectures we need a GCC toolchain for i686.  With [[https://gnu.org/software/guix][GNU Guix]] it’s easy to set up a temporary environment containing just the right tools: the GCC toolchain, make, and Hugs.
  28
  29 #+BEGIN_SRC sh
  30 guix environment --system=i686-linux \
  31                  --ad-hoc gcc-toolchain@4 make hugs
  32 #+END_SRC
  33
  34 * Building the runtime
  35
  36 Now we can configure nhc98 and build the C runtime, which is needed to link the binary objects that nhc98 and GCC produce when compiling Haskell sources.  Configuration is easy:
  37
  38 #+BEGIN_SRC sh
  39 cd /path/to/nhc98-1.22
  40 export NHCDIR=$PWD
  41 ./configure
  42 #+END_SRC
  43
  44 Next, we build the C runtime:
  45
  46 #+BEGIN_SRC sh
  47 cd src/runtime
  48 make
  49 #+END_SRC
  50
  51 This produces binary objects in an architecture-specific directory.  In my case this is =targets/x86_64-Linux=.
  52
  53 * Building the prelude?
  54
  55 The standard library in Haskell is called the prelude.  Most source files of nhc98 depend on the prelude in one way or another.  Although Hugs comes with its own prelude it is of little use for our purposes as components of nhc98 must be linked with a static prelude library object.  Hugs does not provide a suitable object that we could link to.
  56
  57 To build the prelude we need a Haskell compiler that has the same call interface as nhc98.  Interestingly, much of the nhc98 compiler’s user interface is implemented as a shell script, which after extensive argument processing calls =nhc98comp= to translate Haskell source files into C code, and then runs GCC over the C files to create binary objects.  Since we do not have =nhc98comp= at this point, we need to fake it with Hugs (more on that later).
  58
  59 * Detour: cpphs and GreenCard
  60
  61 Unfortunately, this is not the only problem.  Some of the prelude’s source files require pre-processing by a tool called GreenCard, which generates boilerplate FFI code.  Of course, GreenCard is written in Haskell.  Since we cannot build a native GreenCard binary without the native nhc98 prelude library, we need to make GreenCard run on Hugs.  Some of the GreenCard sources require pre-processing with =cpphs=.  Luckily, that’s just a really simple Haskell script, so running it in Hugs is trivial.  We will need =cpphs= later again, so it makes sense to write a script for it.  Let’s call it =hugs-cpphs= and drop it in =${NHCDIR}=.
  62
  63 #+BEGIN_SRC sh
  64 #!/bin/bash
  65
  66 runhugs ${NHCDIR}/src/cpphs/cpphs.hs --noline -D__HASKELL98__ "$@"
  67 #+END_SRC
  68
  69 Make it executable:
  70
  71 #+BEGIN_SRC sh
  72 chmod +x hugs-cpphs
  73 #+END_SRC
  74
  75 Okay, let’s first pre-process the GreenCard sources with =cpphs=.  To do that I ran the following commands:
  76
  77 #+BEGIN_SRC sh
  78 cd ${NHCDIR}/src/greencard
  79
  80 CPPPRE="${NHCDIR}/hugs-cpphs -D__NHC__"
  81
  82 FILES="DIS.lhs \
  83  HandLex.hs \
  84  ParseLib.hs \
  85  HandParse.hs \
  86  FillIn.lhs \
  87  Proc.lhs \
  88  NHCBackend.hs \
  89  NameSupply.lhs \
  90  Process.lhs"
  91
  92 for file in $FILES; do
  93     cp $file $file.original && $CPPPRE $file.original > $file && rm $file.original
  94 done
  95 #+END_SRC
  96
  97 The result is a bunch of GreenCard source files without these pesky CPP pre-processor directives.  Hugs can pre-process sources on the fly, but this makes evaluation orders of magnitude slower.  Pre-processing the sources once before using them repeatedly seems like a better choice.
  98
  99 There is still a minor problem with GreenCard on Hugs.  The GreenCard sources import the module =NonStdTrace=, which depends on built-in prelude functions from nhc98.  Obviously, they are not available when running on Hugs (it has its own prelude implementation), so we need to provide an alternative using just the regular Hugs prelude.  The following snippet creates a file named =src/prelude/NonStd/NonStdTraceBootstrap.hs= with the necessary changes.
 100
 101 #+BEGIN_SRC sh
 102 cd ${NHCDIR}/src/prelude/NonStd/
 103 sed -e 's|NonStdTrace|NonStdTraceBootstrap|' \
 104     -e 's|import PreludeBuiltin||' \
 105     -e 's|_||g' NonStdTrace.hs > NonStdTraceBootstrap.hs
 106 #+END_SRC
 107
 108 Then we change a single line in  =src/greencard/NHCBackend.hs= to make it import =NonStdTraceBootstrap= instead of =NonStdTrace=.
 109
 110 #+BEGIN_SRC sh
 111 cd ${NHCDIR}/src/greencard
 112 sed -i -e 's|NonStdTrace|NonStdTraceBootstrap|' NHCBackend.hs
 113 #+END_SRC
 114
 115 To run GreenCard we still need a driver script.  Let’s call this =hugs-greencard= and place it in =${NHCDIR}=:
 116
 117 #+BEGIN_SRC sh
 118 #!/bin/bash
 119
 120 HUGSDIR="$(dirname $(readlink -f $(which runhugs)))/../"
 121 SEARCH_HUGS=$(printf "${NHCDIR}/src/%s/*:" compiler prelude libraries)
 122
 123 runhugs -98 \
 124         -P${HUGSDIR}/lib/hugs/packages/*:${NHCDIR}/include/*:${SEARCH_HUGS} \
 125         ${NHCDIR}/src/greencard/GreenCard.lhs \
 126         $@
 127 #+END_SRC
 128
 129 Make it executable:
 130
 131 #+BEGIN_SRC sh
 132 cd ${NHCDIR}
 133 chmod +x hugs-greencard
 134 #+END_SRC
 135
 136 * Building the prelude!
 137
 138 Where were we?  Ah, the prelude.  As stated earlier, we need a working replacement for =nhc98comp=, which will be called by the driver script =script/nhc98= (created by the configure script).  Let’s call the replacement =hugs-nhc=, and again we’ll dump it in =${NHCDIR}=.  Here it is in all its glory:
 139
 140 #+BEGIN_SRC sh
 141 #!/bin/bash
 142
 143 # Root directory of Hugs installation
 144 HUGSDIR="$(dirname $(readlink -f $(which runhugs)))/../"
 145
 146 # TODO: "libraries" alone may be sufficient
 147 SEARCH_HUGS=$(printf "${NHCDIR}/src/%s/*:" compiler prelude libraries)
 148
 149 # Filter everything from "+RTS" to "-RTS" from $@ because MainNhc98.hs
 150 # does not know what to do with these flags.
 151 ARGS=""
 152 SKIP="false"
 153 for arg in "$@"; do
 154     if [[ $arg == "+RTS" ]]; then
 155         SKIP="true"
 156     elif [[ $arg == "-RTS" ]]; then
 157         SKIP="false"
 158     elif [[ $SKIP == "false" ]]; then
 159         ARGS="${ARGS} $arg"
 160     fi
 161 done
 162
 163 runhugs -98 \
 164         -P${HUGSDIR}/lib/hugs/packages/*:${SEARCH_HUGS} \
 165         ${NHCDIR}/src/compiler98/MainNhc98.hs \
 166         $ARGS
 167 #+END_SRC
 168
 169 All this does is run Hugs (=runhugs=) with language extensions (=-98=),  ensures that Hugs knows where to look for Hugs and nhc98 modules (=-P=), loads up the compiler’s =main= function, and then passes any arguments other than RTS flags (=$ARGS=) to it.
 170
 171 Let’s also make this executable:
 172
 173 #+BEGIN_SRC sh
 174 cd ${NHCDIR}
 175 chmod +x hugs-nhc
 176 #+END_SRC
 177
 178 The compiler sources contain pre-processor directives, which need to be removed before running =hugs-nhc=.  It would be foolish to let Hugs pre-process the sources at runtime with =-F=.  In my tests it made =hugs-nhc= run slower by an order of magnitude.  Let’s pre-process the sources of the compiler and the libraries it depends on with =hugs-cpphs= (see above):
 179
 180 #+BEGIN_SRC sh
 181 cd ${NHCDIR}
 182 CPPPRE="${NHCDIR}/hugs-cpphs -D__HUGS__"
 183
 184 FILES="src/compiler98/GcodeLowC.hs \
 185  src/libraries/filepath/System/FilePath.hs \
 186  src/libraries/filepath/System/FilePath/Posix.hs"
 187
 188 for file in $FILES; do
 189     cp $file $file.original && $CPPPRE $file.original > $file && rm $file.original
 190 done
 191 #+END_SRC
 192
 193 The compiler’s driver script =script/nhc98= expects to find the executables of =hmake-PRAGMA=, =greencard-nhc98=, and =cpphs= in the architecture-specific lib directory (in my case that’s =${NHCDIR}/lib/x86_64-Linux/=).  They do not exist, obviously, but for two of them we already have scripts to run them on top of Hugs.  =hmake-PRAGMA= does not seem to be very important; replacing it with =cat= appears to be fine.  To pacify the compiler script it’s easiest to just replace a few definitions:
 194
 195 #+BEGIN_SRC sh
 196 cd ${NHCDIR}
 197 sed -i \
 198   -e '0,/^GREENCARD=.*$/s||GREENCARD="$NHC98BINDIR/../hugs-greencard"|' \
 199   -e '0,/^CPPHS=.*$/s||CPPHS="$NHC98BINDIR/../hugs-cpphs -D__NHC__"|' \
 200   -e '0,/^PRAGMA=.*$/s||PRAGMA=cat|' \
 201   script/nhc98
 202 #+END_SRC
 203
 204 Initially, this looked like it would be enough, but half-way through building the prelude Hugs choked when interpreting nhc98 to build a certain module.  After some experimentation it turned out that the =NHC.FFI= module in =src/prelude/FFI/CTypes.hs= is too big for Hugs.  Running nhc98 on that module causes Hugs to abort with an overflow in the control stack.  The fix here is to break up the module to make it easier for nhc98 to build it, which in turn prevents Hugs from doing too much work at once.
 205
 206 Apply this patch:
 207
 208 #+BEGIN_EXAMPLE
 209 From 9eb2a2066eb9f93e60e447aab28479af6c8b9759 Mon Sep 17 00:00:00 2001
 210 From: Ricardo Wurmus <rekado@elephly.net>
 211 Date: Sat, 7 Jan 2017 22:31:41 +0100
 212 Subject: [PATCH] Split up CTypes
 213
 214 This is necessary to avoid a control stack overflow in Hugs when
 215 building the FFI library with nhc98 running on Hugs.
 216 ---
 217  src/prelude/FFI/CStrings.hs     |  2 ++
 218  src/prelude/FFI/CTypes.hs       | 14 --------------
 219  src/prelude/FFI/CTypes1.hs      | 20 ++++++++++++++++++++
 220  src/prelude/FFI/CTypes2.hs      | 22 ++++++++++++++++++++++
 221  src/prelude/FFI/CTypesExtra.hs  |  2 ++
 222  src/prelude/FFI/FFI.hs          |  2 ++
 223  src/prelude/FFI/Makefile        |  8 ++++----
 224  src/prelude/FFI/MarshalAlloc.hs |  2 ++
 225  src/prelude/FFI/MarshalUtils.hs |  2 ++
 226  9 files changed, 56 insertions(+), 18 deletions(-)
 227  create mode 100644 src/prelude/FFI/CTypes1.hs
 228  create mode 100644 src/prelude/FFI/CTypes2.hs
 229
 230 diff --git a/src/prelude/FFI/CStrings.hs b/src/prelude/FFI/CStrings.hs
 231 index 18fdfa9..f1373cf 100644
 232 --- a/src/prelude/FFI/CStrings.hs
 233 +++ b/src/prelude/FFI/CStrings.hs
 234 @@ -23,6 +23,8 @@ module NHC.FFI (
 235
 236  import MarshalArray
 237  import CTypes
 238 +import CTypes1
 239 +import CTypes2
 240  import Ptr
 241  import Word
 242  import Char
 243 diff --git a/src/prelude/FFI/CTypes.hs b/src/prelude/FFI/CTypes.hs
 244 index 18e9d60..942e7a1 100644
 245 --- a/src/prelude/FFI/CTypes.hs
 246 +++ b/src/prelude/FFI/CTypes.hs
 247 @@ -4,11 +4,6 @@ module NHC.FFI
 248           -- Typeable, Storable, Bounded, Real, Integral, Bits
 249           CChar(..),    CSChar(..),  CUChar(..)
 250         , CShort(..),   CUShort(..), CInt(..),    CUInt(..)
 251 -       , CLong(..),    CULong(..),  CLLong(..),  CULLong(..)
 252 -
 253 -         -- Floating types, instances of: Eq, Ord, Num, Read, Show, Enum,
 254 -         -- Typeable, Storable, Real, Fractional, Floating, RealFrac, RealFloat
 255 -       , CFloat(..),   CDouble(..), CLDouble(..)
 256         ) where
 257
 258  import NonStdUnsafeCoerce
 259 @@ -29,12 +24,3 @@ INTEGRAL_TYPE(CShort,Int16)
 260  INTEGRAL_TYPE(CUShort,Word16)
 261  INTEGRAL_TYPE(CInt,Int)
 262  INTEGRAL_TYPE(CUInt,Word32)
 263 -INTEGRAL_TYPE(CLong,Int32)
 264 -INTEGRAL_TYPE(CULong,Word32)
 265 -INTEGRAL_TYPE(CLLong,Int64)
 266 -INTEGRAL_TYPE(CULLong,Word64)
 267 -
 268 -FLOATING_TYPE(CFloat,Float)
 269 -FLOATING_TYPE(CDouble,Double)
 270 --- HACK: Currently no long double in the FFI, so we simply re-use double
 271 -FLOATING_TYPE(CLDouble,Double)
 272 diff --git a/src/prelude/FFI/CTypes1.hs b/src/prelude/FFI/CTypes1.hs
 273 new file mode 100644
 274 index 0000000..81ba0f5
 275 --- /dev/null
 276 +++ b/src/prelude/FFI/CTypes1.hs
 277 @@ -0,0 +1,20 @@
 278 +{-# OPTIONS_COMPILE -cpp #-}
 279 +module NHC.FFI
 280 +       ( CLong(..),    CULong(..),  CLLong(..),  CULLong(..)
 281 +       ) where
 282 +
 283 +import NonStdUnsafeCoerce
 284 +import Int     ( Int8,  Int16,  Int32,  Int64  )
 285 +import Word    ( Word8, Word16, Word32, Word64 )
 286 +import Storable        ( Storable(..) )
 287 +-- import Data.Bits( Bits(..) )
 288 +-- import NHC.SizedTypes
 289 +import Monad   ( liftM )
 290 +import Ptr     ( castPtr )
 291 +
 292 +#include "CTypes.h"
 293 +
 294 +INTEGRAL_TYPE(CLong,Int32)
 295 +INTEGRAL_TYPE(CULong,Word32)
 296 +INTEGRAL_TYPE(CLLong,Int64)
 297 +INTEGRAL_TYPE(CULLong,Word64)
 298 diff --git a/src/prelude/FFI/CTypes2.hs b/src/prelude/FFI/CTypes2.hs
 299 new file mode 100644
 300 index 0000000..7d66242
 301 --- /dev/null
 302 +++ b/src/prelude/FFI/CTypes2.hs
 303 @@ -0,0 +1,22 @@
 304 +{-# OPTIONS_COMPILE -cpp #-}
 305 +module NHC.FFI
 306 +       ( -- Floating types, instances of: Eq, Ord, Num, Read, Show, Enum,
 307 +         -- Typeable, Storable, Real, Fractional, Floating, RealFrac, RealFloat
 308 +       CFloat(..), CDouble(..), CLDouble(..)
 309 +       ) where
 310 +
 311 +import NonStdUnsafeCoerce
 312 +import Int     ( Int8,  Int16,  Int32,  Int64  )
 313 +import Word    ( Word8, Word16, Word32, Word64 )
 314 +import Storable        ( Storable(..) )
 315 +-- import Data.Bits( Bits(..) )
 316 +-- import NHC.SizedTypes
 317 +import Monad   ( liftM )
 318 +import Ptr     ( castPtr )
 319 +
 320 +#include "CTypes.h"
 321 +
 322 +FLOATING_TYPE(CFloat,Float)
 323 +FLOATING_TYPE(CDouble,Double)
 324 +-- HACK: Currently no long double in the FFI, so we simply re-use double
 325 +FLOATING_TYPE(CLDouble,Double)
 326 diff --git a/src/prelude/FFI/CTypesExtra.hs b/src/prelude/FFI/CTypesExtra.hs
 327 index ba3f15b..7cbdcbb 100644
 328 --- a/src/prelude/FFI/CTypesExtra.hs
 329 +++ b/src/prelude/FFI/CTypesExtra.hs
 330 @@ -20,6 +20,8 @@ import Storable       ( Storable(..) )
 331  import Monad   ( liftM )
 332  import Ptr     ( castPtr )
 333  import CTypes
 334 +import CTypes1
 335 +import CTypes2
 336
 337  #include "CTypes.h"
 338
 339 diff --git a/src/prelude/FFI/FFI.hs b/src/prelude/FFI/FFI.hs
 340 index 9d91e57..0c29394 100644
 341 --- a/src/prelude/FFI/FFI.hs
 342 +++ b/src/prelude/FFI/FFI.hs
 343 @@ -217,6 +217,8 @@ import MarshalUtils -- routines for basic marshalling
 344  import MarshalError    -- routines for basic error-handling
 345
 346  import CTypes          -- newtypes for various C basic types
 347 +import CTypes1
 348 +import CTypes2
 349  import CTypesExtra     -- types for various extra C types
 350  import CStrings                -- C pointer to array of char
 351  import CString         -- nhc98-only
 352 diff --git a/src/prelude/FFI/Makefile b/src/prelude/FFI/Makefile
 353 index 99065f8..e229672 100644
 354 --- a/src/prelude/FFI/Makefile
 355 +++ b/src/prelude/FFI/Makefile
 356 @@ -18,7 +18,7 @@ EXTRA_C_FLAGS =
 357  SRCS = \
 358         Addr.hs Ptr.hs FunPtr.hs Storable.hs \
 359         ForeignObj.hs ForeignPtr.hs Int.hs Word.hs \
 360 -       CError.hs CTypes.hs CTypesExtra.hs CStrings.hs \
 361 +       CError.hs CTypes.hs CTypes1.hs CTypes2.hs CTypesExtra.hs CStrings.hs \
 362         MarshalAlloc.hs MarshalArray.hs MarshalError.hs MarshalUtils.hs \
 363         StablePtr.hs
 364
 365 @@ -38,12 +38,12 @@ Word.hs: Word.hs.cpp
 366  # dependencies generated by hmake -Md: (and hacked by MW)
 367  ${OBJDIR}/MarshalError.$O: ${OBJDIR}/Ptr.$O
 368  ${OBJDIR}/MarshalUtils.$O: ${OBJDIR}/Ptr.$O ${OBJDIR}/Storable.$O \
 369 -       ${OBJDIR}/MarshalAlloc.$O ${OBJDIR}/CTypes.$O ${OBJDIR}/CTypesExtra.$O
 370 +       ${OBJDIR}/MarshalAlloc.$O ${OBJDIR}/CTypes.$O ${OBJDIR}/CTypes1.$O ${OBJDIR}/CTypes2.$O ${OBJDIR}/CTypesExtra.$O
 371  ${OBJDIR}/MarshalArray.$O: ${OBJDIR}/Ptr.$O ${OBJDIR}/Storable.$O \
 372         ${OBJDIR}/MarshalAlloc.$O ${OBJDIR}/MarshalUtils.$O
 373 -${OBJDIR}/CTypesExtra.$O: ${OBJDIR}/Int.$O ${OBJDIR}/Word.$O ${OBJDIR}/CTypes.$O
 374 +${OBJDIR}/CTypesExtra.$O: ${OBJDIR}/Int.$O ${OBJDIR}/Word.$O ${OBJDIR}/CTypes.$O ${OBJDIR}/CTypes1.$O ${OBJDIR}/CTypes2.$O
 375  ${OBJDIR}/CTypes.$O: ${OBJDIR}/Int.$O ${OBJDIR}/Word.$O ${OBJDIR}/Storable.$O \
 376 -       ${OBJDIR}/Ptr.$O
 377 +       ${OBJDIR}/Ptr.$O ${OBJDIR}/CTypes1.$O ${OBJDIR}/CTypes2.$O
 378  ${OBJDIR}/CStrings.$O: ${OBJDIR}/MarshalArray.$O ${OBJDIR}/CTypes.$O \
 379         ${OBJDIR}/Ptr.$O ${OBJDIR}/Word.$O
 380  ${OBJDIR}/MarshalAlloc.$O: ${OBJDIR}/Ptr.$O ${OBJDIR}/Storable.$O \
 381 diff --git a/src/prelude/FFI/MarshalAlloc.hs b/src/prelude/FFI/MarshalAlloc.hs
 382 index 34ac7b3..5b43554 100644
 383 --- a/src/prelude/FFI/MarshalAlloc.hs
 384 +++ b/src/prelude/FFI/MarshalAlloc.hs
 385 @@ -14,6 +14,8 @@ import ForeignPtr (FinalizerPtr(..))
 386  import Storable
 387  import CError
 388  import CTypes
 389 +import CTypes1
 390 +import CTypes2
 391  import CTypesExtra (CSize)
 392  import NHC.DErrNo
 393
 394 diff --git a/src/prelude/FFI/MarshalUtils.hs b/src/prelude/FFI/MarshalUtils.hs
 395 index 312719b..bd9d149 100644
 396 --- a/src/prelude/FFI/MarshalUtils.hs
 397 +++ b/src/prelude/FFI/MarshalUtils.hs
 398 @@ -29,6 +29,8 @@ import Ptr
 399  import Storable
 400  import MarshalAlloc
 401  import CTypes
 402 +import CTypes1
 403 +import CTypes2
 404  import CTypesExtra
 405
 406  -- combined allocation and marshalling
 407 --
 408 2.11.0
 409 #+END_EXAMPLE
 410
 411 After all this it’s time for a break.  Run the following commands for a long break:
 412
 413 #+BEGIN_SRC sh
 414 cd ${NHCDIR}/src/prelude
 415 time make NHC98COMP=$NHCDIR/hugs-nhc
 416 #+END_SRC
 417
 418 After the break—it took more than two hours on my laptop—you should see output like this:
 419
 420 #+BEGIN_EXAMPLE
 421 ranlib /path/to/nhc98-1.22/lib/x86_64-Linux/Prelude.a
 422 #+END_EXAMPLE
 423
 424 Congratulations!  You now have a native nhc98 prelude library!
 425
 426 * Building hmake
 427
 428 The compiler and additional Haskell libraries all require a tool called “hmake” to automatically order dependencies, so we’ll try to build it next.  There’s just a small problem with one of the source files: =src/hmake/FileName.hs= contains the name “Niklas Röjemo” and the compiler really does not like the umlaut.  With apologies to Niklas we change the copyright line to appease the compiler.
 429
 430 #+BEGIN_SRC sh
 431 cd $NHCDIR/src/hmake
 432 mv FileName.hs{,.broken}
 433 tr '\366' 'o' < FileName.hs.broken > FileName.hs
 434 rm FileName.hs.broken
 435 NHC98COMP=$NHCDIR/hugs-nhc make HC=$NHCDIR/script/nhc98
 436 #+END_SRC
 437
 438 * To be continued
 439
 440 Unfortunately, the hmake tools are not working.  All of the tools (e.g. =MkConfig=) fail with an early segmentation fault.  There must be an error in the runtime, likely in =src/runtime/Kernel/mutator.c= where bytecode for heap and stack operations is interpreted.  One thing that looks like a problem is statements like this:
 441
 442 #+BEGIN_EXAMPLE
 443 *--sp = (NodePtr) constptr[-HEAPOFFSET(ip[0])];
 444 #+END_EXAMPLE
 445
 446 =constptr= is =NULL=, so this seems to be just pointer arithmetic expressed in array notation.  These errors can be fixed by rewriting the statement to use explicit pointer arithmetic:
 447
 448 #+BEGIN_EXAMPLE
 449 *--sp = (NodePtr) (constptr + (-HEAPOFFSET(ip[0])));
 450 #+END_EXAMPLE
 451
 452 Unfortunately, this doesn’t seem to be enough as there is another segfault in the handling of the =EvalTOS= label.  =IND_REMOVE= is applied to the contents of the stack pointer, which turns out to be =0x10=, which just doesn’t seem right.  =IND_REMOVE= removes indirection by following pointer addresses until the value stored at the given address does not look like an address.  This fails because =0x10= does look like an address—it’s just invalid.  I have enabled a bunch of tracing and debugging features, but I don’t fully understand how the nhc98 runtime is /supposed/ to work.
 453
 454 Judging from mails on the nhc-bugs and nhc-users lists I see that I’m not the only one experiencing segfaults.  [[https://mail.haskell.org/pipermail/nhc-bugs/2005-October/000529.html][This email]] suggests that segfaults are “associated with changes in the way gcc lays out static arrays of bytecodes, e.g. by putting extra padding space between arrays that are supposed to be adjacent.”  I may have to try different compiler flags or an older version of GCC; I only tried with GCC 4.9.4 but the [[http://sources.debian.net/src/nhc98/1.16-15/debian/rules/][Debian package for nhc98]] used version 2.95 or 3.3.
 455
 456 For completeness sake here’s the trace of the failing execution of MkProg:
 457
 458 #+BEGIN_EXAMPLE
 459 (gdb) run
 460 Starting program: /path/to/nhc98-1.22/lib/x86_64-Linux/MkProg
 461 ZAP_ARG_I1              hp=0x80c5010 sp=0x8136a10 fp=0x8136a10 ip=0x8085140
 462 NEEDHEAP_I32    hp=0x80c5010 sp=0x8136a10 fp=0x8136a10 ip=0x8085141
 463 HEAP_CVAL_N1    hp=0x80c5010 sp=0x8136a10 fp=0x8136a10 ip=0x8085142
 464 HEAP_CVAL_I3    hp=0x80c5014 sp=0x8136a10 fp=0x8136a10 ip=0x8085144
 465 HEAP_OFF_N1             hp=0x80c5018 sp=0x8136a10 fp=0x8136a10 ip=0x8085145
 466 PUSH_HEAP               hp=0x80c501c sp=0x8136a10 fp=0x8136a10 ip=0x8085147
 467 HEAP_CVAL_I4    hp=0x80c501c sp=0x8136a0c fp=0x8136a10 ip=0x8085148
 468 HEAP_CVAL_I5    hp=0x80c5020 sp=0x8136a0c fp=0x8136a10 ip=0x8085149
 469 HEAP_OFF_N1             hp=0x80c5024 sp=0x8136a0c fp=0x8136a10 ip=0x808514a
 470 PUSH_CVAL_P1    hp=0x80c5028 sp=0x8136a0c fp=0x8136a10 ip=0x808514c
 471 PUSH_I1             hp=0x80c5028 sp=0x8136a08 fp=0x8136a10 ip=0x808514e
 472 ZAP_STACK_P1    hp=0x80c5028 sp=0x8136a04 fp=0x8136a10 ip=0x808514f
 473 EVAL                hp=0x80c5028 sp=0x8136a04 fp=0x8136a10 ip=0x8085151
 474 eval: evalToS
 475
 476 Program received signal SIGSEGV, Segmentation fault.
 477 0x0804ac27 in run (toplevel=0x80c5008) at mutator.c:425
 478 425             IND_REMOVE(nodeptr);
 479 #+END_EXAMPLE
 480
 481 =ip= is the instruction pointer, which points at the current element in the bytecode stream.  =fp= is probably the frame pointer, =sp= the stack pointer, and =hp= the heap pointer.  The [[https://www.haskell.org/nhc98/implementation-notes/index.html][implementation notes for nhc98]] will probably be helpful in solving this problem.
 482
 483 Anyway, that’s where I’m at so far.   If you are interested in these kinds of problems or other bootstrapping projects, consider joining the efforts of the [[http://bootstrappable.org][Bootstrappable Builds project]]!