question: Does the `conf` directory has to be writable?

[duesee]

I managed to make script-server work with NixOS and my first attempt was to provide the conf as a static set of files. In NixOS, these live in /nix/store. However, the /nix/store is not writable. The error I am getting is "Read-only filesystem" and this happens (apparently) during a migration? It seems that script-server wants to modify the runners folder.

Question: Is my hypothesis correct that the files in conf are modified at runtime? Do you have a suggestion how to separate "config" from "state" or disable migrations altogether since we are always pinning an exact version?

[bugy]

Hi @duesee, I'm not sure if this is possible:

• as you noticed Script Server has migrations. I.e. if some field is renamed between versions or changes from a simple string to a json object. In this case there is an automatic migration when you run a newer version on a conf files created in an old version. And old syntax is replaced with a new one
• Script Server has admin UI, where it's possible to change configuration of the runners
• Script Server allows to schedule script execution. Such schedules are also stored in conf/schedules folder. One can also create schedules manully, without using UI

The only really "static" file is conf.json and one can override its location by using a command line arguments. But even this is questionable, in my vision conf.json should be also configurable via UI.

[duesee]

Understood, thanks! What my use case is: I want to completely auto-deploy the script-server, which means, ideally, that we pin down the whole configuration, version, etc. once and never change it. I feel, we even want to disable the admin interface since the script-server will be used for a self-servicing kiosk so to say.

So, I feel we don't need migrations at all and can (in case of an update) migrate "manually".

Regarding the scheduled execution: This is "state" data, right?

Our workaround here could be to first copy our initial config to /var/lib/script-server/conf and then use this writable directory. But: When we change the initial conf, we would need to replace (parts of?) the migrated conf.

Stupid question: Is the migration so costly that it is better to persist it? Or can the migration run on every startup (when needed)?

[bugy]

Is the migration so costly that it is better to persist it? Or can the migration run on every startup (when needed)?

It's a bit costly and some migrations can be not idempotent. Also it would mean that some migrations should be maintained between releases and consider different update paths
And the cost grows over time with more migrations added.

I want to completely auto-deploy the script-server, which means, ideally, that we pin down the whole configuration, version, etc. once and never change it

Then you can apply the migrations and use this "final" version for auto-deploy, can't you? Since migrations run only once, it's fine to have a read-only folder.

Regarding the scheduled execution: This is "state" data, right?

From your use-case perspective yes (if you "freeze" runners and conf). From my perspective they are similar to runners. But if you don't have scripts with scheduling, this shouldn't be an issue for you, right? You can still use readonly folder for config.
I mean, I was working with different software in the past. And most of them had mutable configs (either manually or through UI/CLI). That's the whole idea of the config files: if they are immutable, they can just effectively live in the code.

[duesee]

Then you can apply the migrations and use this "final" version for auto-deploy, can't you? Since migrations run only once, it's fine to have a read-only folder.

Ow! That is an excellent idea. I think I used one of the provided samples and concluded that having a writable directory is a requirement no matter if a migration is needed since I expected the samples to be "fresh". 'll see how far I get, thank you!

I mean, I was working with different software in the past. And most of them had mutable configs (either manually or through UI/CLI). That's the whole idea of the config files: if they are immutable, they can just effectively live in the code.

To clarify: Immutable doesn't mean config is static. Of course, we want to adapt it (and also do). But here, script-server changes its "config" at runtime. I am not sure I saw a Linux service that modified its config in e.g. /etc recently. If anything these files were always modified by specific tools that were called by an administrator (effectively being "manual editing") -- but I get how script-server is such a tool itself!

Could it be that the word "conf" mislead me? If conf would have been called "database" I would have directly tried to populate it at a writable location. But conf made me think of /etc which is rather unusual to contain frequently changed files.

[bugy]

Could it be that the word "conf" mislead me? If conf would have been called "database" I would have directly tried to populate it at a writable location. But conf made me think of /etc which is rather unusual to contain frequently changed files.

Yeah, database might be a better analogy