Skip to content

1104 mibm higher order central differencing#1614

Open
danieljvickers wants to merge 13 commits into
MFlowCode:masterfrom
danieljvickers:1104-mibm-higher-order-central-differencing
Open

1104 mibm higher order central differencing#1614
danieljvickers wants to merge 13 commits into
MFlowCode:masterfrom
danieljvickers:1104-mibm-higher-order-central-differencing

Conversation

@danieljvickers

Copy link
Copy Markdown
Member

Description

For a while I have been putting off adding higher-order central difference mostly because I did not see an immediate need. However, it appears that the current viscous force calculation in MFC has some significant errors for low Reynolds cases. Increasing the finite-difference order for the derivatives to 4th order appears to significantly reduce this error back within acceptable tolerances. This branch integrates the IB code with the existing finite-difference coefficients in MFC to be utilized in the force calculation loop. This has added the side-benefit of somewhat reducing the amount of code in the s_compute_ib_forces subroutine

Closes #1104

Type of change (delete unused ones)

  • Bug fix
  • New feature
  • Refactor

Testing

I did a significant amount of post-processing analysis of the data to determine the best integration method for low-Reynolds cases, which wound up being

Checklist

Check these like this [x] to indicate which of the below applies.

  • I added or updated tests for new behavior
  • I updated documentation if user-facing behavior changed

See the developer guide for full coding standards.

GPU changes (expand if you modified src/simulation/)
  • GPU results match CPU results
  • Tested on NVIDIA GPU or AMD GPU

@github-actions

github-actions Bot commented Jun 22, 2026

Copy link
Copy Markdown

Claude Code Review

Head SHA: a13521b

Files changed:

  • 7
  • examples/2D_mibm_cylinder_in_cross_flow/case.py
  • examples/2D_mibm_particle_cloud/case.py
  • examples/2D_mibm_shock_cylinder/case.py
  • src/simulation/m_derived_variables.fpp
  • src/simulation/m_global_parameters.fpp
  • src/simulation/m_ibm.fpp
  • src/simulation/m_viscous.fpp

Findings

[Correctness] 3D radial-vector bug in m_ibm.fpp

radial_vector = [x_cc(i), y_cc(j), 0._wp] - [patch_ib(ib_idx)%x_centroid, &
                              & patch_ib(ib_idx)%y_centroid, 0._wp]
if (num_dims == 3) radial_vector(3) = patch_ib(ib_idx)%z_centroid

The initial array constructor correctly sets radial_vector(1:2) to the cell-to-centroid offset. However, in 3D the last line replaces radial_vector(3) with z_centroid alone, instead of z_cc(k) - patch_ib(ib_idx)%z_centroid. The vector from the IB centroid to the grid cell is (x_cc(i) - x_centroid, y_cc(j) - y_centroid, z_cc(k) - z_centroid); the new code gives (x_cc(i) - x_centroid, y_cc(j) - y_centroid, z_centroid). This produces wrong torques for all 3D IB cases. The correct replacement is:

if (num_dims == 3) radial_vector(3) = z_cc(k) - patch_ib(ib_idx)%z_centroid

[Memory] accel_mag/x_accel/y_accel/z_accel allocated under ib but freed only under probe_wrt in m_derived_variables.fpp

s_initialize_derived_variables_module now allocates those four arrays whenever probe_wrt .or. ib:

if (probe_wrt .or. ib) then
    ...
    @:ALLOCATE(accel_mag(0:m, 0:n, 0:p))
    @:ALLOCATE(x_accel(0:m, 0:n, 0:p))
    ...
end if

But s_finalize_derived_variables_module (unchanged) frees them only when probe_wrt:

if (probe_wrt) then
    deallocate (accel_mag, x_accel)
    ...
end if

When ib .and. .not. probe_wrt these arrays are allocated, never used (the only consumer is inside if (probe_wrt) in s_compute_derived_variables), and never freed. The finalize condition should be widened to probe_wrt .or. ib, or the accel allocations should remain guarded by probe_wrt alone and be split out of the shared if block.

@danieljvickers

Copy link
Copy Markdown
Member Author

Fixed the AI review comment

@danieljvickers

Copy link
Copy Markdown
Member Author

I reran a convergence test and compared the force integration to my independent python implementation of all of the drag coefficient for on a 900x900 grid with Re=40 Mach 0.5 cylinder using fd_order: 4:
image

You can see that we now obtain the 4th-order integral. I take this as now working, and will proceed to updating tests before calling this complete.

@sbryngelson

Copy link
Copy Markdown
Member

it certainly seems better, but remind me what we think accounts for the difference between the reference and our result

@danieljvickers

danieljvickers commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

@sbryngelson This is only on a 900x900 grid. I get much less error as I converge towards larger grid sizes in this case. This is to demonstrate that it matches the python solver that I have been using for analysis. I have separate results in my slides showing the accuracy of the python solver. Rerunning at a much higher resolution will just take a few hours, but I can do that if you want me to include it before merge.

@danieljvickers danieljvickers marked this pull request as ready for review June 22, 2026 19:41
@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 69.23077% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.48%. Comparing base (f181a6b) to head (3211746).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/simulation/m_derived_variables.fpp 0.00% 0 Missing and 3 partials ⚠️
src/simulation/m_ibm.fpp 82.35% 0 Missing and 3 partials ⚠️
src/simulation/m_global_parameters.fpp 0.00% 0 Missing and 1 partial ⚠️
src/simulation/m_viscous.fpp 80.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1614      +/-   ##
==========================================
- Coverage   60.49%   60.48%   -0.01%     
==========================================
  Files          83       83              
  Lines       19902    19888      -14     
  Branches     2951     2951              
==========================================
- Hits        12040    12030      -10     
+ Misses       5868     5864       -4     
  Partials     1994     1994              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@danieljvickers

danieljvickers commented Jun 23, 2026

Copy link
Copy Markdown
Member Author

@sbryngelson Did a higher resolution sim over night. This is 3000x3000 (100 grid cells across the cylinder diameter) of the Re=40 Ma=0.1 case
image

MFC computed a drag coefficient of 1.549 compared to the reference value of 1.540 from Canuto D, Taira K. Two-dimensional compressible viscous flow around a circular cylinder. Journal of Fluid Mechanics. 2015;785:349-371. doi:10.1017/jfm.2015.635

@sbryngelson

Copy link
Copy Markdown
Member

This is worth documenting in an example and the readme since it's central to our claim that we have IBM working with the correct forces.

@sbryngelson

Copy link
Copy Markdown
Member

otherwise it seems ready to merge

@danieljvickers

Copy link
Copy Markdown
Member Author

@sbryngelson

Copy link
Copy Markdown
Member

the fd_order input variable is terribly overloaded. It's used for ~3 different purposes now. This seems to be working at the moment, but it can break if you use the right (wrong) parameter set and it's not a great strategy anyway. Can you open a GH issue to disambiguate the various fd_order's we use in the code somehow? FYI this pr looks good, I'll probably just let CI run then merge.

@danieljvickers

Copy link
Copy Markdown
Member Author

I don't think it is as clear-cut as that. When we run with fd_order, MFC on start up instantiated the fd_coeffs_x[yz] arrays that hold 5 or 3 floating point values at every grid cell, which are the coefficients taking in the local dx value.

Separating them gives more flexibility, but setting all 4 FD orders would instantiate a whole lot of memory that is somewhat wasted. The benefit of doing it the way I did was first that I did not add more case file parameters, but mostly that I am piggy-backing off already instantiated memory values.

I think a better refactor would probably be drop the coefficient arrays altogether. They are extremely extremely cheap to compute, so storing them is wasteful. I feel like it is more efficient to compute the coefficient subroutine directly inside the loop to get the coefficient values. But doing this way would mean that we can have different finite difference orders for all 4 different cases in the same case file and that the net memory usage goes down. I do not know exactly how much it costs to compute them, but my intuition and basic reasoning tells me that it is almost zero.

@sbryngelson

Copy link
Copy Markdown
Member

these input parameters are also used by post process in an entirely different manner, e.g., for schlieren i think. that's mostly what i mean. so we have a checker that checks if X postprocess parameter is true and if fd_order > 2 (for example). and setting fd_order the way you do conflicts with that.

the main problem isn't your idea, it's that MFC already overloaded this parameter.

re: compute at runtime vs. store in array. i think keeping in array might make sense because the array is small and used in many places. DRY.

@danieljvickers

Copy link
Copy Markdown
Member Author

Opened the issue and copied the relevant comments. Discussion about this can move there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

MIBM Higher Order Central Differencing

2 participants