MINIFICPP-2860 Add Conan support for the remaining thirdparty libraries#2214
Open
lordgamez wants to merge 1 commit into
Open
MINIFICPP-2860 Add Conan support for the remaining thirdparty libraries#2214lordgamez wants to merge 1 commit into
lordgamez wants to merge 1 commit into
Conversation
- Add Conan support for abseil, protobuf, grpc, gcp, ossp-uuid, bustache, jolt, rpmalloc, kubernetes, azure-sdk, llamacpp, date, winflexbison, and aws-sdk-cpp libraries
lordgamez
force-pushed
the
MINIFICPP-2860
branch
from
July 14, 2026 09:17
d72fa53 to
6f11394
Compare
lordgamez
marked this pull request as ready for review
July 14, 2026 11:26
martinzink
reviewed
Jul 16, 2026
| add_minifi_multi_option(MINIFI_KAFKA_SOURCE "Retrieves librdkafka from provided source" "BUILD;CONAN" "BUILD") | ||
| add_minifi_multi_option(MINIFI_MAGIC_ENUM_SOURCE "Retrieves magic_enum from provided source" "BUILD;CONAN" "BUILD") | ||
| add_minifi_multi_option(MINIFI_OPC_SOURCE "Retrieves open62541 from provided source" "BUILD;CONAN" "BUILD") | ||
| add_minifi_multi_option(MINIFI_OSSP_UUID_SOURCE "Retrieves ossp-uuid from provided source" "BUILD;CONAN" "BUILD") |
Member
There was a problem hiding this comment.
i think we should introduce a deafult flag, so something like this
add_minifi_multi_option(MINIFI_DEFAULT_DEPENDENCY_SOURCE "Retrieves dependencies from provided source by default" "BUILD;CONAN" "BUILD")
add_minifi_multi_option(MINIFI_LZ4_SOURCE "Retrieves lz4 from provided source" "BUILD;CONAN" "${MINIFI_DEFAULT_DEPENDENCY_SOURCE}")
add_minifi_multi_option(MINIFI_LIBCURL_SOURCE "Retrieves LibCURL from provided source" "BUILD;CONAN" "${MINIFI_DEFAULT_DEPENDENCY_SOURCE}")
...
Contributor
There was a problem hiding this comment.
Pull request overview
Adds Conan packaging support for a broad set of remaining third-party dependencies and wires the build system to select between bundled (FetchContent) and Conan-provided sources via new Get* CMake modules. This continues the Conan integration work referenced by MINIFICPP-2860 (and depends on #2212).
Changes:
- Introduces Conan recipes + metadata for multiple third-party libraries (e.g., grpc, google-cloud-cpp, aws-sdk-cpp, azure-sdk-cpp, rpmalloc, ossp-uuid, llama.cpp, kubernetes client, bustache, jolt test resources).
- Updates core/extension CMake to use new
Get*modules and modernizes some dependency/link target usage. - Extends the top-level Conan build (
conanfile.py, bootstrap CLI) with new feature flags/options and required dependencies.
Reviewed changes
Copilot reviewed 78 out of 97 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| thirdparty/rpmalloc/config.yml | Adds rpmalloc Conan recipe version mapping |
| thirdparty/rpmalloc/all/conanfile.py | Adds rpmalloc Conan recipe |
| thirdparty/rpmalloc/all/conandata.yml | Adds rpmalloc source URL/hash |
| thirdparty/rpmalloc/all/CMakeLists.txt | Adds CMake build shim for rpmalloc |
| thirdparty/ossp-uuid/config.yml | Adds ossp-uuid Conan recipe version mapping |
| thirdparty/ossp-uuid/all/patches/ossp-uuid-update-config-guess.patch | Normalizes patch header formatting |
| thirdparty/ossp-uuid/all/patches/ossp-uuid-no-prog.patch | Adds patch to avoid building/installing CLI program |
| thirdparty/ossp-uuid/all/patches/ossp-uuid-mac-fix.patch | Adds patch to improve MAC handling/portability |
| thirdparty/ossp-uuid/all/conanfile.py | Adds ossp-uuid Conan recipe |
| thirdparty/ossp-uuid/all/conandata.yml | Adds ossp-uuid sources + patch list |
| thirdparty/llamacpp/config.yml | Adds llama.cpp Conan recipe version mapping |
| thirdparty/llamacpp/all/patches/mtmd-fix.patch | Adds llama.cpp patch for mtmd + include fix |
| thirdparty/llamacpp/all/conanfile.py | Adds llama.cpp Conan recipe |
| thirdparty/llamacpp/all/conandata.yml | Adds llama.cpp sources + patches |
| thirdparty/kubernetes-client-c/config.yml | Adds kubernetes-client-c Conan recipe version mapping |
| thirdparty/kubernetes-client-c/all/patches/remove-findpackage.patch | Adds patch to remove upstream find_package usage |
| thirdparty/kubernetes-client-c/all/conanfile.py | Adds kubernetes C client Conan recipe |
| thirdparty/kubernetes-client-c/all/conandata.yml | Adds kubernetes-client-c sources |
| thirdparty/jolt-tests/config.yml | Adds jolt test resources recipe version mapping |
| thirdparty/jolt-tests/all/conanfile.py | Adds Conan package for jolt test resources |
| thirdparty/jolt-tests/all/conandata.yml | Adds jolt sources URL/hash |
| thirdparty/grpc/fix-memory-request-missing-string-include.patch | Removes obsolete patch file |
| thirdparty/grpc/fix-glob-missing-algorithm-include.patch | Removes obsolete patch file |
| thirdparty/grpc/config.yml | Adds grpc Conan recipe version mapping |
| thirdparty/grpc/all/target_info/grpc_1.82.0.yml | Adds grpc target metadata used for packaging |
| thirdparty/grpc/all/patches/fix-protobuf-find-package.patch | Adds grpc patch for protobuf discovery |
| thirdparty/grpc/all/patches/fix-msvc-auto-return-type-template-arg.patch | Adds grpc patch for MSVC compilation issue |
| thirdparty/grpc/all/conanfile.py | Adds grpc Conan recipe |
| thirdparty/grpc/all/conandata.yml | Adds grpc sources + patches |
| thirdparty/grpc/all/conan_cmake_project_include.cmake | Adds grpc CMake project include workaround |
| thirdparty/grpc/all/cmake/grpc_plugin_template.cmake.in | Adds template for grpc plugin imported targets |
| thirdparty/google-cloud-cpp/remove_installation.patch | Removes obsolete patch file |
| thirdparty/google-cloud-cpp/config.yml | Adds google-cloud-cpp Conan recipe version mapping |
| thirdparty/google-cloud-cpp/all/patches/remove-find_package.patch | Adds patch adjusting upstream dependency discovery |
| thirdparty/google-cloud-cpp/all/patches/nlohmann_lib_as_interface.patch | Adds patch to enforce nlohmann_json usage |
| thirdparty/google-cloud-cpp/all/patches/c++23_fixes.patch | Adds patch for C++23 compatibility |
| thirdparty/google-cloud-cpp/all/conanfile.py | Adds google-cloud-cpp Conan recipe (storage-focused) |
| thirdparty/google-cloud-cpp/all/conandata.yml | Adds google-cloud-cpp sources + patches |
| thirdparty/bustache/config.yml | Adds bustache Conan recipe version mapping |
| thirdparty/bustache/all/patches/remove_installs.patch | Adds patch to remove upstream install rules |
| thirdparty/bustache/all/patches/fix-deprecated-literal-operator.patch | Adds patch for newer compiler literal operator |
| thirdparty/bustache/all/patches/add-append.patch | Adds patch for append helper needed by MiNiFi |
| thirdparty/bustache/all/conanfile.py | Adds bustache Conan recipe |
| thirdparty/bustache/all/conandata.yml | Adds bustache sources + patches |
| thirdparty/azure-sdk-cpp/fix-managed-identity.patch | Removes obsolete patch file |
| thirdparty/azure-sdk-cpp/config.yml | Adds azure-sdk-cpp Conan recipe version mapping |
| thirdparty/azure-sdk-cpp/all/patches/wil.patch | Adds patch to avoid upstream wil find_package |
| thirdparty/azure-sdk-cpp/all/patches/include-cinttypes-for-uint8_t-gcc15-fix.patch | Adds patch for missing includes |
| thirdparty/azure-sdk-cpp/all/patches/fix-openssl-helper.patch | Adds patch for openssl helper implementation |
| thirdparty/azure-sdk-cpp/all/conanfile.py | Adds azure-sdk-cpp Conan recipe |
| thirdparty/azure-sdk-cpp/all/conandata.yml | Adds azure-sdk-cpp sources + patches |
| thirdparty/aws-sdk-cpp/config.yml | Adds aws-sdk-cpp Conan recipe version mapping |
| thirdparty/aws-sdk-cpp/all/patches/s2n.patch | Adds patch to use OpenSSL targets for s2n |
| thirdparty/aws-sdk-cpp/all/patches/fix-finding-s2n.patch | Adds patch to force aws_use_package(s2n) |
| thirdparty/aws-sdk-cpp/all/patches/dll-export-injection.patch | Adds patch for Windows DLL export behavior |
| thirdparty/aws-sdk-cpp/all/patches/aws-c-cal.patch | Adds patch to adjust OpenSSL discovery/install bits |
| thirdparty/aws-sdk-cpp/all/conanfile.py | Adds aws-sdk-cpp Conan recipe |
| thirdparty/aws-sdk-cpp/all/conandata.yml | Adds aws-sdk-cpp source + patches list |
| extensions/standard-processors/tests/CMakeLists.txt | Switches to GetJoltTests include |
| extensions/llamacpp/CMakeLists.txt | Switches to GetLlamaCpp + updates link targets |
| extensions/kubernetes/CMakeLists.txt | Switches to GetKubernetesClientC include |
| extensions/grafana-loki/CMakeLists.txt | Switches to GetGrpc + shared proto generation module |
| extensions/gcp/tests/CMakeLists.txt | Adjusts test include/link logic to support Conan GTest |
| extensions/gcp/CMakeLists.txt | Switches to GetGoogleCloudCpp include |
| extensions/bustache/CMakeLists.txt | Switches to GetBustache include |
| extensions/azure/CMakeLists.txt | Switches to GetAzureSdkCpp include |
| extensions/aws/CMakeLists.txt | Switches to GetAwsSdkCpp include |
| conanfile.py | Adds new Conan options and requirements for the new libs |
| CMakeLists.txt | Switches some deps to Get* modules (rpmalloc, ossp-uuid, date) |
| cmake/WholeArchive.cmake | Enhances whole-archive handling for Conan interface targets |
| cmake/Protobuf.cmake | Updates bundled protobuf version |
| cmake/MiNiFiOptions.cmake | Adds new multi-options for selecting BUILD vs CONAN sources |
| cmake/LlamaCpp.cmake | Fixes patch path and adds alias targets |
| cmake/KubernetesClientC.cmake | Fixes patch path for Kubernetes client C |
| cmake/Grpc.cmake | Updates grpc tag and patch paths; adds gRPC::grpc++ alias |
| cmake/GoogleCloudCpp.cmake | Fixes patch paths; exposes test gtest link libs |
| cmake/GetWinFlexBison.cmake | Adds BUILD/CONAN switch for winflexbison |
| cmake/GetRpMalloc.cmake | Adds BUILD/CONAN switch for rpmalloc |
| cmake/GetOSSPUUID.cmake | Adds BUILD/CONAN switch for ossp-uuid |
| cmake/GetLlamaCpp.cmake | Adds BUILD/CONAN switch for llama.cpp |
| cmake/GetKubernetesClientC.cmake | Adds BUILD/CONAN switch for kubernetes client C |
| cmake/GetJoltTests.cmake | Adds BUILD/CONAN switch for jolt test resources |
| cmake/GetGrpc.cmake | Adds BUILD/CONAN switch for grpc |
| cmake/GetGoogleCloudCpp.cmake | Adds BUILD/CONAN switch for google-cloud-cpp (+ GTest handling) |
| cmake/GetDate.cmake | Adds BUILD/CONAN switch for date (+ tzdata staging on Windows) |
| cmake/GetBustache.cmake | Adds BUILD/CONAN switch for bustache |
| cmake/GetAzureSdkCpp.cmake | Adds BUILD/CONAN switch for azure SDK |
| cmake/GetAwsSdkCpp.cmake | Adds BUILD/CONAN switch for aws SDK |
| cmake/GenerateGrafanaLokiProto.cmake | Centralizes Loki proto generation |
| cmake/ExpressionLanguage.cmake | Switches Windows winflexbison setup to GetWinFlexBison |
| cmake/Date.cmake | Removes Windows tzdata staging logic (moved to GetDate) |
| bootstrap/cli.py | Extends Conan option mapping + updates conan settings flags |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+18
to
+24
| if(MINIFI_LLAMACPP_SOURCE STREQUAL "CONAN") | ||
| message("Using Conan to install LlamaCpp") | ||
| find_package(llamacpp REQUIRED) | ||
| elseif(MINIFI_LLAMACPP_SOURCE STREQUAL "BUILD") | ||
| message("Using CMake to build LlamaCpp from source") | ||
| include(LlamaCpp) | ||
| endif() |
Comment on lines
+93
to
+97
| if self.settings.os == "Windows": | ||
| if not self.options.transport_curl and not self.options.get_safe("transport_winhttp"): | ||
| raise ConanInvalidConfiguration("On Windows, HTTP Transport options: transport_winhttp or transport_curl must be enabled.") | ||
| elif not self.options.transport_curl: | ||
| raise ConanInvalidConfiguration("The HTTP Transport option transport_curl must be enabled.") |
Comment on lines
+46
to
+47
| default_options = {"shared": False, "fPIC": True} | ||
|
|
Comment on lines
+1
to
+3
| versions: | ||
| "b8944": | ||
| folder: "all" |
Comment on lines
+1
to
+3
| versions: | ||
| "0.14.0": | ||
| folder: "all" |
Comment on lines
+18
to
+27
| if(MINIFI_GRPC_SOURCE STREQUAL "CONAN") | ||
| message("Using Conan to install gRPC") | ||
| find_package(protobuf CONFIG REQUIRED) | ||
| find_package(gRPC CONFIG REQUIRED) | ||
| set(PROTOBUF_INCLUDE_DIR "${protobuf_INCLUDE_DIR}" CACHE STRING "" FORCE) | ||
| set(GRPC_INCLUDE_DIR "${gRPC_INCLUDE_DIR}" CACHE STRING "" FORCE) | ||
| elseif(MINIFI_GRPC_SOURCE STREQUAL "BUILD") | ||
| message("Using CMake to build gRPC from source") | ||
| include(Grpc) | ||
| endif() |
Comment on lines
+25
to
+29
| find_program(LOKI_PROTOC_EXECUTABLE | ||
| NAMES protoc | ||
| PATHS "${protobuf_INCLUDE_DIR}/../bin" | ||
| NO_DEFAULT_PATH | ||
| REQUIRED) |
Comment on lines
+45
to
+48
| DEPENDS | ||
| ${LOKI_PROTOC_EXECUTABLE} | ||
| gRPC::grpc_cpp_plugin | ||
| ${CMAKE_CURRENT_SOURCE_DIR}/protos/grafana-loki-push.proto |
Comment on lines
+34
to
+35
| file(COPY ${CMAKE_CURRENT_SOURCE_DIR}/thirdparty/cldr-common-38.1/common/supplemental/windowsZones.xml | ||
| DESTINATION ${CMAKE_BINARY_DIR}/tzdata) |
Comment on lines
+44
to
+46
| install(FILES ${CMAKE_CURRENT_SOURCE_DIR}/thirdparty/cldr-common-38.1/common/supplemental/windowsZones.xml | ||
| DESTINATION tzdata | ||
| COMPONENT bin) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/MINIFICPP-2860
Depends on #2212
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
Has your PR been rebased against the latest commit within the target branch (typically main)?
Is your initial contribution a single, squashed commit?
For code changes:
For documentation related changes:
Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.