Skip to content

Explicitly instantiate function return values that are template classes #1024

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions src/bindgen/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,15 @@ pub struct ExportConfig {
pub renaming_overrides_prefixing: bool,
/// Mangling configuration.
pub mangle: MangleConfig,
/// Whether to instantiate the monomorphs of template types used as function return values. This
/// is needed for C compatibility, because otherwise compilers warn (`-Wreturn-type-c-linkage`
/// on gcc/clang) or even reject (MSVC) those function definitions. The compensation is made by
/// emitting a single struct with one field for each monomorphized type. The emitted wrapper
/// struct's name can optionally be overridden by [`return_value_monomorphs_struct_name`].
pub instantiate_return_value_monomorphs: bool,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I think this is kind of a workaround... But yeah it's not pretty. I don't think you need a struct or so tho, you can just explicitly instantiate them, right?

See here for example.

Copy link
Contributor Author

@scovich scovich May 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay responding... misrouted github notification email...

From the PR description:

We take the dummy struct approach because explicit instantiation changes semantics of C++ templates and can cause linker errors if multiple compilation units do it -- not good for a general header file.

The approach linked above would violate the One Definition Rule (ODR), if included by multiple different compilation units of the same project. For example, according to https://en.cppreference.com/w/cpp/language/class_template:

An explicit instantiation definition forces instantiation of the class, struct, or union they refer to. It may appear in the program anywhere after the template definition, and for a given argument-list, is only allowed to appear once in the entire program, no diagnostic required.

The "no diagnostic required" part is annoying -- the compiler/linker isn't required to say anything, the resulting binary just has undefined behavior. See e.g.
https://stackoverflow.com/questions/45120323/why-c-linker-is-silent-about-odr-violation
https://stackoverflow.com/questions/21534435/separate-compilation-and-template-explicit-instantiation

C++11 added support for "explicit instantiation declarations" by prepending the extern keyword, but that just pushes the problem somewhere else:

An explicit instantiation declaration (an extern template) skips implicit instantiation step: the code that would otherwise cause an implicit instantiation instead uses the explicit instantiation definition provided elsewhere (resulting in link errors if no such instantiation exists).

By "using" a specific template instantiation inside a struct definition, the compiler implicitly instantiates the template instead, which makes its definition available without violating the ODR -- a special case that would otherwise cause header file class definitions to violate ODR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(also updated the PR description with some of this info, since it wasn't clear before)

/// Overrides the struct name to use when [`instantiate_return_value_monomorphs`] is enabled
/// (ignored otherwise). If not specified, the default is `__cbindgen_return_value_monomorphs`.
pub return_value_monomorphs_struct_name: Option<String>,
}

/// Mangling-specific configuration.
Expand Down
6 changes: 2 additions & 4 deletions src/bindgen/ir/documentation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

use crate::bindgen::utilities::SynAttributeHelpers;

#[derive(Debug, Clone)]
#[derive(Debug, Clone, Default)]
pub struct Documentation {
pub doc_comment: Vec<String>,
}
Expand All @@ -27,8 +27,6 @@ impl Documentation {
}

pub fn none() -> Self {
Documentation {
doc_comment: Vec::new(),
}
Self::default()
}
}
10 changes: 9 additions & 1 deletion src/bindgen/ir/enumeration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ use crate::bindgen::ir::{
use crate::bindgen::language_backend::LanguageBackend;
use crate::bindgen::library::Library;
use crate::bindgen::mangle;
use crate::bindgen::monomorph::Monomorphs;
use crate::bindgen::monomorph::{Monomorphs, ReturnValueMonomorphs};
use crate::bindgen::rename::{IdentifierType, RenameRule};
use crate::bindgen::reserved;
use crate::bindgen::writer::{ListType, SourceWriter};
Expand Down Expand Up @@ -317,6 +317,14 @@ impl Enum {
repr.style != ReprStyle::C
}

pub fn find_return_value_monomorphs(&self, monomorphs: &mut ReturnValueMonomorphs<'_>) {
for v in &self.variants {
if let VariantBody::Body { ref body, .. } = v.body {
body.find_return_value_monomorphs(monomorphs);
}
}
}

pub fn add_monomorphs(&self, library: &Library, out: &mut Monomorphs) {
if self.is_generic() {
return;
Expand Down
15 changes: 13 additions & 2 deletions src/bindgen/ir/function.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@ use syn::ext::IdentExt;
use crate::bindgen::config::{Config, Language};
use crate::bindgen::declarationtyperesolver::DeclarationTypeResolver;
use crate::bindgen::dependencies::Dependencies;
use crate::bindgen::ir::{AnnotationSet, Cfg, Documentation, GenericPath, Path, Type};
use crate::bindgen::ir::{
AnnotationSet, Cfg, Documentation, GenericParams, GenericPath, Path, Type,
};
use crate::bindgen::library::Library;
use crate::bindgen::monomorph::Monomorphs;
use crate::bindgen::monomorph::{Monomorphs, ReturnValueMonomorphs};
use crate::bindgen::rename::{IdentifierType, RenameRule};
use crate::bindgen::reserved;
use crate::bindgen::utilities::IterHelpers;
Expand Down Expand Up @@ -47,6 +49,12 @@ impl Function {
attrs: &[syn::Attribute],
mod_cfg: Option<&Cfg>,
) -> Result<Function, String> {
if let Ok(GenericParams(generics)) = GenericParams::load(&sig.generics) {
if !generics.is_empty() {
return Err("Generic functions are not supported".to_owned());
}
}

let mut args = sig.inputs.iter().try_skip_map(|x| x.as_argument())?;
if sig.variadic.is_some() {
args.push(FunctionArgument {
Expand Down Expand Up @@ -129,6 +137,9 @@ impl Function {
}
}

pub fn find_return_value_monomorphs(&self, monomorphs: &mut ReturnValueMonomorphs<'_>) {
monomorphs.handle_function(&self.ret, self.args.iter().map(|arg| &arg.ty));
}
pub fn add_monomorphs(&self, library: &Library, out: &mut Monomorphs) {
self.ret.add_monomorphs(library, out);
for arg in &self.args {
Expand Down
13 changes: 9 additions & 4 deletions src/bindgen/ir/structure.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ use crate::bindgen::ir::{
};
use crate::bindgen::library::Library;
use crate::bindgen::mangle;
use crate::bindgen::monomorph::Monomorphs;
use crate::bindgen::monomorph::{Monomorphs, ReturnValueMonomorphs};
use crate::bindgen::rename::{IdentifierType, RenameRule};
use crate::bindgen::reserved;
use crate::bindgen::utilities::IterHelpers;
Expand Down Expand Up @@ -168,9 +168,14 @@ impl Struct {

/// Attempts to convert this struct to a typedef (only works for transparent structs).
pub fn as_typedef(&self) -> Option<Typedef> {
match self.fields.first() {
Some(field) if self.is_transparent => Some(Typedef::new_from_struct_field(self, field)),
_ => None,
let field = self.fields.first()?;
self.is_transparent
.then(|| Typedef::new_from_struct_field(self, field))
}

pub fn find_return_value_monomorphs(&self, monomorphs: &mut ReturnValueMonomorphs<'_>) {
for field in &self.fields {
field.ty.find_return_value_monomorphs(monomorphs, false);
}
}

Expand Down
18 changes: 17 additions & 1 deletion src/bindgen/ir/ty.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ use crate::bindgen::declarationtyperesolver::DeclarationTypeResolver;
use crate::bindgen::dependencies::Dependencies;
use crate::bindgen::ir::{GenericArgument, GenericParams, GenericPath, Path};
use crate::bindgen::library::Library;
use crate::bindgen::monomorph::Monomorphs;
use crate::bindgen::monomorph::{Monomorphs, ReturnValueMonomorphs};
use crate::bindgen::utilities::IterHelpers;

#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]
Expand Down Expand Up @@ -739,6 +739,22 @@ impl Type {
self.add_dependencies_ignoring_generics(&GenericParams::default(), library, out)
}

pub fn find_return_value_monomorphs(
&self,
monomorphs: &mut ReturnValueMonomorphs<'_>,
is_return_value: bool,
) {
match self {
Type::Ptr { ty, .. } => ty.find_return_value_monomorphs(monomorphs, false),
Type::Path(generic) => monomorphs.handle_return_value_path(generic, is_return_value),
Type::Primitive(_) => {}
Type::Array(ty, _) => ty.find_return_value_monomorphs(monomorphs, false),
Type::FuncPtr { ret, args, .. } => {
monomorphs.handle_function(ret, args.iter().map(|(_, arg)| arg))
}
}
}

pub fn add_monomorphs(&self, library: &Library, out: &mut Monomorphs) {
match *self {
Type::Ptr { ref ty, .. } => {
Expand Down
11 changes: 10 additions & 1 deletion src/bindgen/ir/typedef.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ use crate::bindgen::ir::{
};
use crate::bindgen::library::Library;
use crate::bindgen::mangle;
use crate::bindgen::monomorph::Monomorphs;
use crate::bindgen::monomorph::{Monomorphs, ReturnValueMonomorphs};

/// A type alias that is represented as a C typedef
#[derive(Debug, Clone)]
Expand Down Expand Up @@ -102,6 +102,15 @@ impl Typedef {
}
}

pub fn find_return_value_monomorphs(
&self,
monomorphs: &mut ReturnValueMonomorphs<'_>,
is_return_value: bool,
) {
self.aliased
.find_return_value_monomorphs(monomorphs, is_return_value);
}

pub fn add_monomorphs(&self, library: &Library, out: &mut Monomorphs) {
// Generic structs can instantiate monomorphs only once they've been
// instantiated. See `instantiate_monomorph` for more details.
Expand Down
8 changes: 7 additions & 1 deletion src/bindgen/ir/union.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ use crate::bindgen::ir::{
};
use crate::bindgen::library::Library;
use crate::bindgen::mangle;
use crate::bindgen::monomorph::Monomorphs;
use crate::bindgen::monomorph::{Monomorphs, ReturnValueMonomorphs};
use crate::bindgen::rename::{IdentifierType, RenameRule};
use crate::bindgen::utilities::IterHelpers;

Expand Down Expand Up @@ -100,6 +100,12 @@ impl Union {
}
}

pub fn find_return_value_monomorphs(&self, monomorphs: &mut ReturnValueMonomorphs<'_>) {
for field in &self.fields {
field.ty.find_return_value_monomorphs(monomorphs, false);
}
}

pub fn add_monomorphs(&self, library: &Library, out: &mut Monomorphs) {
// Generic unions can instantiate monomorphs only once they've been
// instantiated. See `instantiate_monomorph` for more details.
Expand Down
38 changes: 35 additions & 3 deletions src/bindgen/library.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,11 @@ use crate::bindgen::config::{Config, Language, SortKey};
use crate::bindgen::declarationtyperesolver::DeclarationTypeResolver;
use crate::bindgen::dependencies::Dependencies;
use crate::bindgen::error::Error;
use crate::bindgen::ir::{Constant, Enum, Function, Item, ItemContainer, ItemMap};
use crate::bindgen::ir::{OpaqueItem, Path, Static, Struct, Typedef, Union};
use crate::bindgen::monomorph::Monomorphs;
use crate::bindgen::ir::{
Constant, Enum, Function, Item, ItemContainer, ItemMap, OpaqueItem, Path, Static, Struct,
Typedef, Union,
};
use crate::bindgen::monomorph::{Monomorphs, ReturnValueMonomorphs};
use crate::bindgen::ItemType;

#[derive(Debug, Clone)]
Expand Down Expand Up @@ -81,6 +83,12 @@ impl Library {

let mut dependencies = Dependencies::new();

if self.config.language == Language::Cxx
&& self.config.export.instantiate_return_value_monomorphs
{
self.instantiate_return_value_monomorphs(&mut dependencies);
}

for function in &self.functions {
function.add_dependencies(&self, &mut dependencies);
}
Expand Down Expand Up @@ -447,4 +455,28 @@ impl Library {
x.mangle_paths(&monomorphs);
}
}

fn instantiate_return_value_monomorphs(&mut self, dependencies: &mut Dependencies) {
let mut monomorphs = ReturnValueMonomorphs::new(self);
self.structs
.for_all_items(|x| x.find_return_value_monomorphs(&mut monomorphs));
self.unions
.for_all_items(|x| x.find_return_value_monomorphs(&mut monomorphs));
self.enums
.for_all_items(|x| x.find_return_value_monomorphs(&mut monomorphs));
self.typedefs
.for_all_items(|x| x.find_return_value_monomorphs(&mut monomorphs, false));
for x in &self.functions {
x.find_return_value_monomorphs(&mut monomorphs);
}

let struct_name = match self.config.export.return_value_monomorphs_struct_name {
Some(ref name) => name,
_ => "__cbindgen_return_value_monomorphs",
};
if let Some((struct_name, struct_def)) = monomorphs.into_struct(struct_name) {
self.structs.try_insert(struct_def);
struct_name.add_dependencies(self, dependencies);
}
}
}
112 changes: 110 additions & 2 deletions src/bindgen/monomorph.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */

use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use std::iter::FromIterator as _;
use std::mem;

use crate::bindgen::ir::{
Enum, GenericArgument, GenericPath, Item, OpaqueItem, Path, Struct, Typedef, Union,
Documentation, Enum, Field, GenericArgument, GenericPath, Item, ItemContainer, OpaqueItem,
Path, Struct, Type, Typedef, Union,
};
use crate::bindgen::library::Library;

Expand Down Expand Up @@ -145,3 +147,109 @@ impl Monomorphs {
mem::take(&mut self.enums)
}
}

/// A helper for collecting all function return value momomorphs -- template types returned by
/// functions that can lead to compilation warnings/errors if not explicitly instantiated.
pub struct ReturnValueMonomorphs<'a> {
library: &'a Library,
monomorphs: HashSet<GenericPath>,
}

impl<'a> ReturnValueMonomorphs<'a> {
pub fn new(library: &'a Library) -> Self {
Self {
library,
monomorphs: HashSet::new(),
}
}

/// Resolve a typedef that is a function return value, specializing it first if needed.
fn handle_return_value_typedef(&mut self, typedef: Typedef, generic: &GenericPath) {
if typedef.is_generic() {
let args = generic.generics();
let aliased = &typedef.aliased;
let mappings = typedef.generic_params.call(typedef.path.name(), args);
let aliased = aliased.specialize(&mappings);
aliased.find_return_value_monomorphs(self, true);
} else {
typedef.find_return_value_monomorphs(self, true);
}
}

/// Once we find a function return type, what we do with it depends on the type of item it
/// resolves to. Typedefs need to be resolved recursively, while generic structs, unions, and
/// enums are captured in the set of return value monomorphs.
pub fn handle_return_value_path(&mut self, generic: &GenericPath, is_return_value: bool) {
if !is_return_value {
return;
}

for item in self.library.get_items(generic.path()).into_iter().flatten() {
match item {
// Constants and statics cannot be function return types
ItemContainer::Constant(_) | ItemContainer::Static(_) => {}
// Opaque items cannot be instantiated (doomed to compilation failure)
ItemContainer::OpaqueItem(_) => {}
ItemContainer::Typedef(t) => self.handle_return_value_typedef(t, generic),
ItemContainer::Union(_) | ItemContainer::Enum(_) => {
if !generic.generics().is_empty() {
self.monomorphs.insert(generic.clone());
}
}
ItemContainer::Struct(s) => {
if let Some(t) = s.as_typedef() {
self.handle_return_value_typedef(t, generic);
} else if !generic.generics().is_empty() {
self.monomorphs.insert(generic.clone());
}
}
}
}
}

/// Whenever we encounter a function (or function pointer), we need to check whether its return
/// value is an instantiated generic type (monomorph).
pub fn handle_function<'i>(&mut self, ret: &Type, args: impl IntoIterator<Item = &'i Type>) {
ret.find_return_value_monomorphs(self, true);
for arg in args.into_iter() {
arg.find_return_value_monomorphs(self, false);
}
}

/// Emit all instantiated return value monomorphs as fields of a dummy struct, which silences
/// warnings and errors on several compilers.
pub fn into_struct(self, struct_name: &str) -> Option<(Type, Struct)> {
if self.monomorphs.is_empty() {
return None;
}

// Sort the output so that the struct remains stable across runs (tests want that).
let mut monomorphs = Vec::from_iter(self.monomorphs);
monomorphs.sort();
let fields = monomorphs
.into_iter()
.enumerate()
.map(|(i, path)| Field::from_name_and_type(format!("field{}", i), Type::Path(path)))
.collect();
let doc_comment = vec![
" Dummy struct emitted by cbindgen to avoid compiler warnings/errors about",
" return type C linkage for template types returned by value from functions",
];
let doc_comment = doc_comment.into_iter().map(Into::into).collect();
let struct_name = GenericPath::new(Path::new(struct_name), vec![]);
let struct_def = Struct::new(
struct_name.path().clone(),
Default::default(), // no generic params
fields,
false, // no tag field
false, // not an enum body
None, // no special alignment requirements
false, // not transparent
None, // no conf
Default::default(), // no annotations
Documentation { doc_comment },
);
let struct_name = Type::Path(struct_name);
Some((struct_name, struct_def))
}
}
6 changes: 6 additions & 0 deletions tests/expectations/const_generics_char.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ struct TakeUntil {
uintptr_t point;
};

/// Dummy struct emitted by cbindgen to avoid compiler warnings/errors about
/// return type C linkage for template types returned by value from functions
struct __cbindgen_return_value_monomorphs {
TakeUntil<0> field0;
};

extern "C" {

TakeUntil<0> until_nul(const uint8_t *start, uintptr_t len);
Expand Down
Loading
Loading