FSharp fsx scripts as a project's viable starting point

My initial plan was to publish a post explaining why I love developing with F#. But the majority of reader of the FsAdvent probably already love F# and don’t need a reminder why, so I switched plan to a recent experience I had with starting out an exploration project with .fsx scripts and migrating to a compiled project.

Here’s what we will cover:

What are fsx scripts🔗

.fsx scripts are scripts written in F#. The very same language is available to write scripts that you can execute with dotnet fsi myscript.fsx.

There are additional constructs to load libraries or other scripts, as no .fsproj file is loaded (fspro files are where compiled project list dependencies). To load a library, you use #r line this: #r "nuget: Newtonsoft.Json". And the script also has access to an fsi object giving for example access to the script arguments via fsi.CommandLineArgs. But apart from this, it’s all standard F#.

Why use fsx?🔗

Starting with .fsx scripts allows for very rapid iterations when exploring opportunities. You can put all code in one file, including dependencies references. It is also very easy to deploy: copy the script to a host or container with the dotnet sdk and you’re ready to run it. No need to include all the resources it needs in its fsproj, compile it, copy it over with all the generated dlls. Deploying is just copying files and running the script.

Any limitations?🔗

You cannot load framework refs in F# scripts, which means you can’t load ASP.Net like you can load a nuget package. There’s a workaround though if you need to use ASP.Net in a fsx script. For exploration code you probably would be happy using a lighter alternative like Suave, but ASP.Net is what you need, there’s a blog post exploring its use in fsx scripts.

Another downside is the startup overhead. .fsx script are run with dotnet fsi, which is meant to be an interactive REPL and does a lot of additional things needed for that specific use, but useless if you just want to execute the script. I would think having a dedicated tool for running scripts would be great, but the devs have probably more important stuff to look at. Especially as for simple scripts, there might be solutions available (see below).

Running tests with F# scripts is not that easy.

It also felt like the developer experience in Neovim was also somewhat lesser than with a compiled project, though I’m probably in the minority using other editors than VS or VSCode.

Main lesson🔗

Developing the solution as .fsx scripts was particularly smooth, even when splitting the code in multiple files. The only point of attention I identified is to load all library files in one file . If you have your mainScript.fsx and library files lib/A.fsx which is a dependency of lib/B.fsx which itself is a dependency of lib/C.fsx, do not #load "lib/A.fsx" in B.fsx and #load "lib/B.fsx" in C.fsx. Place all #load directives in your main script (or even better, at the cost of having an additional file, in a specific file loaded by your main script):

  #load lib/A.fsx"
  #load "lib/B.fsx"
  #load "lib/C.fsx"

If you don’t follow this, you can end up in a mess of scattered #load that lead to the definition of the same type in multiple modules generated by fsi that will prevent your scripts from running. The error reported in that case is of the form

  error FS0001: This expression was expected to have type 'FSI_0001.MyModule.Mytype'
but here has type 'FSI_0002.MyModule.Mytype'

I didn’t see the same requirements with #r directives. However, if you don’t group your #r directives, you might end up with different versions being loaded by different scripts.

In conclusion, grouping all #load and #r directives seems to be a good practice.

Migrating from existing fsx scripts to a compile project🔗

Why and when🔗

This will depend on the situation as someone on the forum reported having more than 10 000 lines in fsx scripts! In my case, it occured much sooner. My code was really exploratory, and deployed in a container running 5 .fsx script, and which was setup to restart if any of the scripts stopped. This was the intended setup, but the startup time of the container became a problem, also because it implied a downtime when deploying a new version. That’s why we switched to a compiled project for the same code. The compile time is moved to the container image building step, and the container starts up much faster.

Migration procedure🔗

If you have a project of .fsx scripts, it’s still easy to convert to a compiled project.

Migrating from the code in .fsx files to a compiled project should be particularly smooth:

1. Rename you files from .fsx extension to .fs.
2. Then create an empty .fsproj file. This example shows the file for a script MyProgram.fsx that was renamed MyProgram.fs:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="MyProgram.fs" />
  </ItemGroup>
</Project>

3. For each #load directive in the scripts and which you remove for the compiled version, add a <Compile> entry before MyProgram.fs. In this example the script has the following loads:

  #load "lib/Shared.fs"
  #load "lib/Queue.fs"
  #load "lib/Index.fs"

which results in this fsproj file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="lib/Shared.fs" />
    <Compile Include="lib/Queue.fs" />
    <Compile Include="lib/Index.fs" />
    <Compile Include="generateIndex.fs" />
  </ItemGroup>
</Project>

4. For each #r "nuget:..." directive, you add the package in the fsproj file (and remove the #r directive from the code). For example for the package FSharp.SystemTextJson at version 1.3.13, you will find on the package’s nuget page the command to add it with the dotnet cli (under the .NET CLI tab):

dotnet add package FSharp.SystemTextJson --version 1.3.13

You will also find the XML to add in your fsproj file under the PackageReference tab. Both ways of adding the dependency result in this file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="lib/Shared.fs" />
    <Compile Include="lib/Queue.fs" />
    <Compile Include="lib/Index.fs" />
    <Compile Include="generateIndex.fs" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="FSharp.SystemTextJson" Version="1.3.13" />
  </ItemGroup>
</Project>

At this time, you can run dotnet build. The only errors I got reported was when I used the fsi object to access the script’s arguments, which can easily be done with System.Environment.GetCommandLineArgs.

Maintaining both scripts and compilation options🔗

After migrating my project, I wondered what would be needed to maintain a project that can both be compiled and executed an .fsx script. And F# actually provides all that’s needed to achieve this as we’ll see below.

Why🔗

When you decide to go for a compiled project, you might already have developed some automation depending on the .fsx scripts. It would probably be easier to migrate gradually.

You might also have a much faster development cycle and feedback loop with scripts so that you want to keep it available.

Another reason to keep both options can be if you want to distribute executables. There are (experimental?) projects such as fflat and FSharpPacker which let you generate executables from an .fsx script. I tested this with fflat, as FSharpPacker only supports dotnet 6 didn’t work out of the box with dotnet 8 at the time I wrote this post, but the author has swiftly fixed it once I had reported it.

The main appeal of fflat is its ease of distribution and fast startup time, but note you don’t get an F# stack trace in case of a crash. As an illustration, here is the startup time reported by Suave when calling dotnet run webui.fsproj:

  INF Smooth! Suave listener started in 27.666ms with binding 0.0.0.0:8080

The time reported when compiling the project with dotnet publish webui.fsproj or dotnet build webui.fsproj is basically the same. Running an executable generated with fflat though reports this:

  INF Smooth! Suave listener started in .804ms with binding 0.0.0.0:8080

More than 30 times faster! This blog post however is not about analysing those numbers, there’s probably a lot to be analysed and tweaked. It just shows that some situations might warrant maintaining a project as both an .fsx project and a compiled project. We’ll now see how to achieve this.

File extensions impact🔗

Before we see how to proceed, here are the things to know about file naming relevant to our goal:

1. dotnet fsi will only execute .fsx files, and will refuse to execute files with the .fs extension.
2. In .fsx files, it is possible to #load files with a .fs extension.
3. .fsproj files accept to compile files with the .fsx extension.
4. #load and #r directives can only be used in files with the .fsx extension

With this information, here is how I converted a project of .fsx files in a project that is also compilable (via a .fsproj file).

How🔗

There are 4 steps:

File rename🔗

I renamed all my lib files with the .fs extension, but I left my main program file with the .fsx extension. This renaming is optional, but I do it for two reasons, namely to:

1. prevent the use of #load and #r in lib files, preventing erroneously using these outside of the loader script below.
2. follow the convention of F# file naming for most files of the project.

Only the file that will be executed with dotnet fsi needs to retain the .fsx extension.

Loader script🔗

I created a loader.fsx file where I placed all my #r and #load directives. This files has this content:

#r "nuget: Suave, 2.6.2"
#r "nuget: System.Data.SQLite, 1.0.119"
#r "nuget: DbFun.Core, 1.1.0"
#r "nuget: FSharp.SystemTextJson, 1.3.13"
#r "nuget: FsHttp"
#r "nuget: NATS.Net, 2.5.2"
#r "nuget: FSharp.Control.TaskSeq"

#load "db.fs"
#load "Shared.fs"
#load "Queue.fs"

fsproj file🔗

Similarly, I created a .fsproj file referencing the same dependencies and fsharp files. The .fs files are referenced in the same order in both files (if you don’t know, this order matters as code in a file only knows what has been defined in file before it. This is often seen as annoying by beginners, but is really easy to work with and avoids cyclic dependencies). Here’s that file:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <Compile Include="lib/db.fs" />
    <Compile Include="lib/Shared.fs" />
    <Compile Include="lib/Queue.fs" />
    <Compile Include="webui.fsx" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="DbFun.Core" Version="1.1.0" />
    <PackageReference Include="FSharp.Control.TaskSeq" Version="0.4.0" />
    <PackageReference Include="FSharp.SystemTextJson" Version="1.3.13" />
    <PackageReference Include="FsHttp" Version="14.5.1" />
    <PackageReference Include="NATS.Net" Version="2.5.2" />
    <PackageReference Include="System.Data.SQLite" Version="1.0.119" />
    <PackageReference Include="Suave" Version="2.6.2" />
  </ItemGroup>

</Project>

Load if interactive🔗

The key functionality enabling these dual projects, compiled and scripted, is a compiler directive to indicate code that is only executed when running the code interactively. That directive is #if INTERACTIVE.

As we have put all #load directives in the file loader.fsx, we can simply conditionally load that file in our Program.fsx:

#if INTERACTIVE
#load "lib/loader.fsx"
#endif

This enables the project to be compiled with dotnet build myProject.fsproj and to be run with dotnet fsi ./Program.fsx, as well as generate a executable with fflat ./Program.fsx. Maintaining both possibilities while the project expands requires a bit of attention but is not cumbersome: both the .fsproj and the loader.fsx files have to be kept in sync. That’s all that’s needed.

Conclusion🔗

.fsx scripts give F# developers additional possibilities. Starting with a script gives you a rapid feedback loop, with all information regarding dependencies in the same file as your F# code. If your project grows and scripts are not a relevant approach anymore, the cost of migrating away is very low and can easily be compensated by the rapid feedback loop you had at the start of the project. It is not interesting for all projects, but their value is undeniable for exploratory code.