Disambiguate `fd_order` case file parameter

[danieljvickers]

In the recent addition of higher-order finite differencing for immersed boundaries from #1614, the fd_order case file parameter has begun to be overloaded in terms of its use. 4 different case file settings all use the same finite difference order. This means that we are somewhat less flexible and the term appears more ambiguous to users.

It would be helpful to consider a refactor of the finite difference scheme for paramters like probes, hypo-elasticity, immersed boundaries, MHD, post-processing, etc. In particular, the finite difference order used in all of these terms should be disambiguated,

The goal seems to be separate the use cases of this in pre_process, simulation, post_process to be distinct

[danieljvickers]

Relevant chunks of conversation from #1614 comments:

@sbryngelson

the fd_order input variable is terribly overloaded. It's used for ~3 different purposes now. This seems to be working at the moment, but it can break if you use the right (wrong) parameter set and it's not a great strategy anyway. Can you open a GH issue to disambiguate the various fd_order's we use in the code somehow? FYI this pr looks good, I'll probably just let CI run then merge.

@danieljvickers

I don't think it is as clear-cut as that. When we run with fd_order, MFC on start up instantiated the fd_coeffs_x[yz] arrays that hold 5 or 3 floating point values at every grid cell, which are the coefficients taking in the local dx value.

Separating them gives more flexibility, but setting all 4 FD orders would instantiate a whole lot of memory that is somewhat wasted. The benefit of doing it the way I did was first that I did not add more case file parameters, but mostly that I am piggy-backing off already instantiated memory values.

I think a better refactor would probably be drop the coefficient arrays altogether. They are extremely extremely cheap to compute, so storing them is wasteful. I feel like it is more efficient to compute the coefficient subroutine directly inside the loop to get the coefficient values. But doing this way would mean that we can have different finite difference orders for all 4 different cases in the same case file and that the net memory usage goes down. I do not know exactly how much it costs to compute them, but my intuition and basic reasoning tells me that it is almost zero.

@sbryngelson

these input parameters are also used by post process in an entirely different manner, e.g., for schlieren i think. that's mostly what i mean. so we have a checker that checks if X postprocess parameter is true and if fd_order > 2 (for example). and setting fd_order the way you do conflicts with that.

the main problem isn't your idea, it's that MFC already overloaded this parameter.

re: compute at runtime vs. store in array. i think keeping in array might make sense because the array is small and used in many places. DRY.