FSharp fsx scripts as a project's viable starting point
2024-12-03 technology
My initial plan was to publish a post explaining why I love developing with F#. But the majority of reader of the FsAdvent probably already love F# and don’t need a reminder why, so I switched plan to a recent experience I had with starting out an exploration project with .fsx
scripts and migrating to a compiled project.
Here’s what we will cover:
What are fsx scripts🔗
.fsx
scripts are scripts written in F#. The very same language is available to write scripts that you can execute with dotnet fsi myscript.fsx
.
There are additional constructs to load libraries or other scripts, as no .fsproj
file is loaded (fspro
files are where compiled project list dependencies).
To load a library, you use #r
line this: #r "nuget: Newtonsoft.Json"
.
And the script also has access to an fsi
object giving for example access to the script arguments via fsi.CommandLineArgs
.
But apart from this, it’s all standard F#.
Why use fsx?🔗
Starting with .fsx
scripts allows for very rapid iterations when exploring opportunities. You can put all code in one file, including dependencies references.
It is also very easy to deploy: copy the script to a host or container with the dotnet sdk and you’re ready to run it.
No need to include all the resources it needs in its fsproj
, compile it, copy it over with all the generated dlls. Deploying is just copying files and running the script.
Any limitations?🔗
You cannot load framework refs in F# scripts, which means you can’t load ASP.Net like you can load a nuget package. There’s a workaround though if you need to use ASP.Net in a fsx script. For exploration code you probably would be happy using a lighter alternative like Suave, but ASP.Net is what you need, there’s a blog post exploring its use in fsx scripts.
Another downside is the startup overhead. .fsx
script are run with dotnet fsi
, which is meant to be an interactive REPL and does a lot of additional things needed for that specific use, but useless if you just want to execute the script.
I would think having a dedicated tool for running scripts would be great, but the devs have probably more important stuff to look at.
Especially as for simple scripts, there might be solutions available (see below).
Running tests with F# scripts is not that easy.
It also felt like the developer experience in Neovim was also somewhat lesser than with a compiled project, though I’m probably in the minority using other editors than VS or VSCode.
Main lesson🔗
Developing the solution as .fsx
scripts was particularly smooth, even when
splitting the code in multiple files. The only point of attention I identified
is to load all library files in one file . If you have
your mainScript.fsx
and library files lib/A.fsx
which is a dependency of
lib/B.fsx
which itself is a dependency of lib/C.fsx
, do not #load "lib/A.fsx"
in B.fsx
and #load "lib/B.fsx"
in C.fsx
.
Place all #load
directives in your main script (or even better, at the cost
of having an additional file, in a specific file loaded by your main script):
#load lib/A.fsx"
#load "lib/B.fsx"
#load "lib/C.fsx"
If you don’t follow this,
you can end up in a mess of scattered #load
that lead to the definition of
the same type in multiple modules generated by fsi that will prevent your
scripts from running. The error reported in that case is of the form
error FS0001: This expression was expected to have type 'FSI_0001.MyModule.Mytype'
but here has type 'FSI_0002.MyModule.Mytype'
I didn’t see the same requirements with #r
directives. However, if you don’t
group your #r
directives, you might end up with different versions being
loaded by different scripts.
In conclusion, grouping all #load
and #r
directives seems to be a good practice.
Migrating from existing fsx scripts to a compile project🔗
Why and when🔗
This will depend on the situation as someone on the forum reported having more than 10 000 lines in fsx scripts!
In my case, it occured much sooner. My code was really exploratory, and deployed in a container running 5 .fsx
script, and which was setup to restart if any of the scripts stopped.
This was the intended setup, but the startup time of the container became a problem, also because it implied a downtime when deploying a new version.
That’s why we switched to a compiled project for the same code. The compile time is moved to the container image building step, and the container starts up much faster.
Migration procedure🔗
If you have a project of .fsx
scripts, it’s still easy to convert to a compiled project.
Migrating from the code in .fsx
files to a compiled project should be particularly smooth:
-
Rename you files from
.fsx
extension to.fs
. -
Then create an empty
.fsproj
file. This example shows the file for a scriptMyProgram.fsx
that was renamedMyProgram.fs
:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="MyProgram.fs" />
</ItemGroup>
</Project>
-
For each
#load
directive in the scripts and which you remove for the compiled version, add a<Compile>
entry beforeMyProgram.fs
. In this example the script has the following loads:
#load "lib/Shared.fs"
#load "lib/Queue.fs"
#load "lib/Index.fs"
which results in this fsproj
file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="lib/Shared.fs" />
<Compile Include="lib/Queue.fs" />
<Compile Include="lib/Index.fs" />
<Compile Include="generateIndex.fs" />
</ItemGroup>
</Project>
-
For each
#r "nuget:..."
directive, you add the package in thefsproj
file (and remove the#r
directive from the code). For example for the packageFSharp.SystemTextJson
at version1.3.13
, you will find on the package’s nuget page the command to add it with the dotnet cli (under the.NET CLI
tab):
dotnet add package FSharp.SystemTextJson --version 1.3.13
You will also find the XML to add in your fsproj file
under the PackageReference
tab.
Both ways of adding the dependency result in this file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="lib/Shared.fs" />
<Compile Include="lib/Queue.fs" />
<Compile Include="lib/Index.fs" />
<Compile Include="generateIndex.fs" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="FSharp.SystemTextJson" Version="1.3.13" />
</ItemGroup>
</Project>
At this time, you can run dotnet build
. The only errors I got reported was when I used the fsi
object to access the script’s arguments,
which can easily be done with System.Environment.GetCommandLineArgs
.
Maintaining both scripts and compilation options🔗
After migrating my project, I wondered what would be needed to maintain a project that can both be compiled
and executed an .fsx
script. And F# actually provides all that’s needed to achieve this as we’ll see below.
Why🔗
When you decide to go for a compiled project, you might already have developed some automation depending on the .fsx
scripts.
It would probably be easier to migrate gradually.
You might also have a much faster development cycle and feedback loop with scripts so that you want to keep it available.
Another reason to keep both options can be if you want to distribute executables.
There are (experimental?) projects such as fflat and FSharpPacker which let you generate executables from an .fsx
script.
I tested this with fflat, as FSharpPacker only supports dotnet 6 didn’t work out of the box with dotnet 8 at the time I wrote this post, but the author has swiftly fixed it once I had reported it.
The main appeal of fflat is its ease of distribution and fast startup time, but note you don’t get an F# stack trace in case of a crash.
As an illustration, here is the startup time reported by Suave when calling dotnet run webui.fsproj
:
INF Smooth! Suave listener started in 27.666ms with binding 0.0.0.0:8080
The time reported when compiling the project with dotnet publish webui.fsproj
or dotnet build webui.fsproj
is basically the same.
Running an executable generated with fflat though reports this:
INF Smooth! Suave listener started in .804ms with binding 0.0.0.0:8080
More than 30 times faster! This blog post however is not about analysing those numbers, there’s probably a lot to be analysed and tweaked. It just shows that some
situations might warrant maintaining a project as both an .fsx
project and a compiled project. We’ll now see how to achieve this.
File extensions impact🔗
Before we see how to proceed, here are the things to know about file naming relevant to our goal:
-
dotnet fsi
will only execute.fsx
files, and will refuse to execute files with the.fs
extension. -
In
.fsx
files, it is possible to#load
files with a.fs
extension. -
.fsproj
files accept to compile files with the.fsx
extension. -
#load
and#r
directives can only be used in files with the.fsx
extension
With this information, here is how I converted a project of .fsx
files in a project that is also compilable (via a .fsproj
file).
How🔗
There are 4 steps:
File rename🔗
I renamed all my lib files with the .fs
extension, but I left my main program file with the .fsx
extension.
This renaming is optional, but I do it for two reasons, namely to:
-
prevent the use of
#load
and#r
in lib files, preventing erroneously using these outside of the loader script below. - follow the convention of F# file naming for most files of the project.
Only the file that will be executed with dotnet fsi
needs to retain the .fsx
extension.
Loader script🔗
I created a loader.fsx
file where I placed all my #r
and #load
directives. This files has this content:
#r "nuget: Suave, 2.6.2"
#r "nuget: System.Data.SQLite, 1.0.119"
#r "nuget: DbFun.Core, 1.1.0"
#r "nuget: FSharp.SystemTextJson, 1.3.13"
#r "nuget: FsHttp"
#r "nuget: NATS.Net, 2.5.2"
#r "nuget: FSharp.Control.TaskSeq"
#load "db.fs"
#load "Shared.fs"
#load "Queue.fs"
fsproj file🔗
Similarly, I created a .fsproj
file referencing the same dependencies and
fsharp files. The .fs
files are referenced in the same order in both files
(if you don’t know, this order matters as code in a file only knows what has
been defined in file before it. This is often seen as annoying by beginners,
but is really easy to work with and avoids cyclic dependencies). Here’s that file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<Compile Include="lib/db.fs" />
<Compile Include="lib/Shared.fs" />
<Compile Include="lib/Queue.fs" />
<Compile Include="webui.fsx" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="DbFun.Core" Version="1.1.0" />
<PackageReference Include="FSharp.Control.TaskSeq" Version="0.4.0" />
<PackageReference Include="FSharp.SystemTextJson" Version="1.3.13" />
<PackageReference Include="FsHttp" Version="14.5.1" />
<PackageReference Include="NATS.Net" Version="2.5.2" />
<PackageReference Include="System.Data.SQLite" Version="1.0.119" />
<PackageReference Include="Suave" Version="2.6.2" />
</ItemGroup>
</Project>
Load if interactive🔗
The key functionality enabling these dual projects, compiled and scripted, is a compiler directive
to indicate code that is only executed when running the code interactively. That directive is #if INTERACTIVE
.
As we have put all #load
directives in the file loader.fsx
, we can simply conditionally load
that file in our Program.fsx
:
#if INTERACTIVE
#load "lib/loader.fsx"
#endif
This enables the project to be compiled with dotnet build myProject.fsproj
and to be run with dotnet fsi ./Program.fsx
,
as well as generate a executable with fflat ./Program.fsx
.
Maintaining both possibilities while the project expands requires a bit of attention but is not cumbersome:
both the .fsproj
and the loader.fsx
files have to be kept in sync. That’s all that’s needed.
Conclusion🔗
.fsx
scripts give F# developers additional possibilities. Starting with a
script gives you a rapid feedback loop, with all information regarding
dependencies in the same file as your F# code. If your project grows and
scripts are not a relevant approach anymore, the cost of migrating away is very
low and can easily be compensated by the rapid feedback loop you had at the start of the project.
It is not interesting for all projects, but their value is undeniable for exploratory code.