Skip to content

Commit

Permalink
Update calling-c-and-fortran docs
Browse files Browse the repository at this point in the history 
* Formatting, wording updates
  • Loading branch information
@kmsquire
kmsquire committed Apr 12, 2019
1 parent 5ce94d8 commit f7d3abe
Showing 1 changed file with 101 additions and 90 deletions.
191 changes: 101 additions & 90 deletions doc/src/manual/calling-c-and-fortran-code.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,12 @@ The code to be called must be available as a shared library. Most C and Fortran
compiled as shared libraries already, but if you are compiling the code yourself using GCC (or
Clang), you will need to use the `-shared` and `-fPIC` options. The machine instructions generated
by Julia's JIT are the same as a native C call would be, so the resulting overhead is the same
as calling a library function from C code. (Non-library function calls in both C and Julia can
be inlined and thus may have even less overhead than calls to shared library functions. When both
libraries and executables are generated by LLVM, it is possible to perform whole-program optimizations
that can even optimize across this boundary, but Julia does not yet support that. In the future,
however, it may do so, yielding even greater performance gains.)
as calling a library function from C code. [^1]

Shared libraries and functions are referenced by a tuple of the form `(:function, "library")`
or `("function", "library")` where `function` is the C-exported function name. `library` refers
to the shared library name: shared libraries available in the (platform-specific) load path will
be resolved by name, and if necessary a direct path may be specified.
or `("function", "library")` where `function` is the C-exported function name, and `library` refers
to the shared library name. Shared libraries available in the (platform-specific) load path will
be resolved by name. The full path to the library may also be specified.

A function name may be used alone in place of the tuple (just `:function` or `"function"`). In
this case the name is resolved within the current process. This form can be used to call C library
Expand All @@ -37,30 +33,33 @@ arrays and other mutable objects which are normally heap-allocated, but also to
scalar values such as integers and floats which are normally stack-allocated and
commonly passed in registers when using C or Julia calling conventions.

Finally, you can use [`ccall`](@ref) to actually generate a call to the library function. Arguments
to [`ccall`](@ref) are as follows:
Finally, you can use [`ccall`](@ref) to actually generate a call to the library function. The arguments
to [`ccall`](@ref) are:

1. A `(:function, "library")` pair, which must be written as a literal constant,
1. A `(:function, "library")` pair,

OR

a `:function` name symbol or `"function"` name string, which is resolved in the
current process,
a `:function` name symbol or `"function"` name string,

OR

a function pointer (for example, from `dlsym`).

2. Return type (see below for mapping the declared C type to Julia)
2. The function's return type

* This argument will be evaluated at compile-time, when the containing method is defined.
3. A tuple of input types, corresponding to the function signature

3. A tuple of input types. The input types must be written as a literal tuple, not a tuple-valued
variable or expression.
4. The actual argument values to be passed to the function, if any; each is a separate parameter.

* This argument will be evaluated at compile-time, when the containing method is defined.

4. The following arguments, if any, are the actual argument values passed to the function.
!!! note
The `(:function, "library")` pair, return type, and input types must be literal constants
(i.e., they can't be variables, but see [Non-constant Function Specifications](@ref) below).

The remaining parameters are evaulated at compile time.

!!! note
See below for how to [map C types to Julia types](@ref mapping-c-types-to-julia).

As a complete but simple example, the following calls the `clock` function from the standard C
library:
Expand Down Expand Up @@ -164,20 +163,20 @@ typedef returntype (*functiontype)(argumenttype, ...)
```
The macro [`@cfunction`](@ref) generates the C-compatible function pointer for a call to a
Julia function. Arguments to [`@cfunction`](@ref) are as follows:
Julia function. The arguments to [`@cfunction`](@ref) are:
1. A Julia Function
2. Return type
2. The return type
3. A literal tuple of input types
Like ccall, all of these arguments will be evaluated at compile-time, when the containing method is defined.
As with `ccall`, all of these arguments will be evaluated at compile-time when the containing method is defined.
Currently, only the platform-default C calling convention is supported. This means that
`@cfunction`-generated pointers cannot be used in calls where WINAPI expects `stdcall`
function on 32-bit windows, but can be used on WIN64 (where `stdcall` is unified with the
C calling convention).
A classic example is the standard C library `qsort` function, declared as:
As a classic example, consider the standard C library `qsort` function, declared as:
```c
void qsort(void *base, size_t nmemb, size_t size,
Expand All @@ -187,10 +186,11 @@ void qsort(void *base, size_t nmemb, size_t size,
The `base` argument is a pointer to an array of length `nmemb`, with elements of `size` bytes
each. `compare` is a callback function which takes pointers to two elements `a` and `b` and returns
an integer less/greater than zero if `a` should appear before/after `b` (or zero if any order
is permitted). Now, suppose that we have a 1d array `A` of values in Julia that we want to sort
is permitted).

Now, suppose that we have a 1d array `A` of values in Julia that we want to sort
using the `qsort` function (rather than Julia's built-in `sort` function). Before we worry about
calling `qsort` and passing arguments, we need to write a comparison function that works for some
arbitrary objects (which define `<`):
calling `qsort` and passing arguments, we need to write a comparison function:

```jldoctest mycompare
julia> function mycompare(a, b)::Cint
Expand All @@ -199,8 +199,8 @@ julia> function mycompare(a, b)::Cint
mycompare (generic function with 1 method)
```

Notice that we have to be careful about the return type: `qsort` expects a function returning
a C `int`, so we annotate the return type of the function to be sure it returns a `Cint`.
``qsort`` expects a comparison function that return a C ``int``, so we annotate the return type
to be ``Cint``.

In order to pass this function to C, we obtain its address using the macro `@cfunction`:

Expand Down Expand Up @@ -234,23 +234,24 @@ julia> A
```

As can be seen, `A` is changed to the sorted array `[-2.7, 1.3, 3.1, 4.4]`. Note that Julia
knows how to convert an array into a `Ptr{Cdouble}`, how to compute the size of a type in bytes
(identical to C's `sizeof` operator), and so on. For fun, try inserting a `println("mycompare($a, $b)")`
line into `mycompare`, which will allow you to see the comparisons that `qsort` is performing
(and to verify that it is really calling the Julia function that you passed to it).
[takes care of converting the array to a `Ptr{Cdouble}`](@ref automatic-type-conversion)), computing
the size of the element type in bytes, and so on.

## Mapping C Types to Julia
For fun, try inserting a `println("mycompare($a, $b)")` line into `mycompare`, which will allow
you to see the comparisons that `qsort` is performing (and to verify that it is really calling
the Julia function that you passed to it).

## [Mapping C Types to Julia](@id mapping-c-types-to-julia)

It is critical to exactly match the declared C type with its declaration in Julia. Inconsistencies
can cause code that works correctly on one system to fail or produce indeterminate results on
a different system.

Note that no C header files are used anywhere in the process of calling C functions: you are responsible
for making sure that your Julia types and call signatures accurately reflect those in the C header
file. (The [Clang package](https://github.com/ihnorton/Clang.jl) can be used to auto-generate
Julia code from a C header file.)
file.[^2]

### Auto-conversion:
### [Automatic Type Conversion](@id automatic-type-conversion)

Julia automatically inserts calls to the [`Base.cconvert`](@ref) function to convert each argument
to the specified type. For example, the following call:
Expand All @@ -259,26 +260,16 @@ to the specified type. For example, the following call:
ccall((:foo, "libfoo"), Cvoid, (Int32, Float64), x, y)
```

will behave as if the following were written:
will behave as if the following were written[^3]:

```julia
ccall((:foo, "libfoo"), Cvoid, (Int32, Float64),
Base.unsafe_convert(Int32, Base.cconvert(Int32, x)),
Base.unsafe_convert(Float64, Base.cconvert(Float64, y)))
Base.convert(Int32, x), Base.convert(Float64, y))
```

[`Base.cconvert`](@ref) normally just calls [`convert`](@ref), but can be defined to return an
arbitrary new object more appropriate for passing to C.
This should be used to perform all allocations of memory that will be accessed by the C code.
For example, this is used to convert an `Array` of objects (e.g. strings) to an array of pointers.

[`Base.unsafe_convert`](@ref) handles conversion to [`Ptr`](@ref) types. It is considered unsafe because
converting an object to a native pointer can hide the object from the garbage collector, causing
it to be freed prematurely.

### Type Correspondences:
### Type Correspondences

First, a review of some relevant Julia type terminology:
First, let's review some relevant Julia type terminology:

| Syntax / Keyword | Example | Description |
|:----------------------------- |:------------------------------------------- |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
Expand Down Expand Up @@ -347,7 +338,7 @@ This can help for writing portable code (and remembering that an `int` in C is n
an `Int` in Julia).


**System Independent:**
**System Independent Types**

| C name | Fortran name | Standard Julia Alias | Julia Base Type |
|:------------------------------------------------------- |:------------------------ |:-------------------- |:-------------------------------------------------------------------------------------------------------------- |
Expand Down Expand Up @@ -388,7 +379,7 @@ to skip the check, you can use `Ptr{UInt8}` as the argument type. `Cstring` can
the [`ccall`](@ref) return type, but in that case it obviously does not introduce any extra
checks and is only meant to improve readability of the call.

**System-dependent:**
**System Dependent Types**

| C name | Standard Julia Alias | Julia Base Type |
|:--------------- |:-------------------- |:-------------------------------------------- |
Expand Down Expand Up @@ -471,7 +462,7 @@ checks and is only meant to improve readability of the call.
!!! note
A C function declared to return `Cvoid` will return the value `nothing` in Julia.

### Struct Type correspondences
### Struct Type Correspondences

Composite types, aka `struct` in C or `TYPE` in Fortran90 (or `STRUCTURE` / `RECORD` in some variants
of F77), can be mirrored in Julia by creating a `struct` definition with the same
Expand All @@ -485,24 +476,27 @@ are not possible in the translation to Julia.

Packed structs and union declarations are not supported by Julia.

You can get a near approximation of a `union` if you know, a priori, the field that will have
You can get an approximation of a `union` if you know, a priori, the field that will have
the greatest size (potentially including padding). When translating your fields to Julia, declare
the Julia field to be only of that type.

Arrays of parameters can be expressed with `NTuple`:
Arrays of parameters can be expressed with `NTuple`. For example, this struct in C,

in C:
```c
struct B {
int A[3];
};

b_a_2 = B.A[2];
```
in Julia:

can be written in Julia as

```julia
struct B
A::NTuple{3, Cint}
end

b_a_2 = B.A[3] # note the difference in indexing (1-based in Julia, 0-based in C)
```

Expand Down Expand Up @@ -602,7 +596,7 @@ In Julia code wrapping calls to external C routines, ordinary (non-pointer) data
to be of type `T` inside the [`ccall`](@ref), as they are passed by value. For C code accepting
pointers, [`Ref{T}`](@ref) should generally be used for the types of input arguments, allowing the use
of pointers to memory managed by either Julia or C through the implicit call to [`Base.cconvert`](@ref).
In contrast, pointers returned by the C function called should be declared to be of output type
In contrast, pointers returned by the C function called should be declared to be of output type
[`Ptr{T}`](@ref), reflecting that the memory pointed to is managed by C only. Pointers contained in C
structs should be represented as fields of type `Ptr{T}` within the corresponding Julia struct
types designed to mimic the internal structure of corresponding C structs.
Expand Down Expand Up @@ -718,37 +712,7 @@ ccall(:foo, Cvoid, (Ref{Cint}, Ref{Cfloat}), width, range)
Upon return, the contents of `width` and `range` can be retrieved (if they were changed by `foo`)
by `width[]` and `range[]`; that is, they act like zero-dimensional arrays.

### Special Reference Syntax for ccall (deprecated):

The `&` syntax is deprecated, use the `Ref{T}` argument type instead.

A prefix `&` is used on an argument to [`ccall`](@ref) to indicate that a pointer to a scalar
argument should be passed instead of the scalar value itself (required for all Fortran function
arguments, as noted above). The following example computes a dot product using a BLAS function.

```julia
function compute_dot(DX::Vector{Float64}, DY::Vector{Float64})
@assert length(DX) == length(DY)
n = length(DX)
incx = incy = 1
product = ccall((:ddot_, "libLAPACK"),
Float64,
(Ref{Int32}, Ptr{Float64}, Ref{Int32}, Ptr{Float64}, Ref{Int32}),
n, DX, incx, DY, incy)
return product
end
```

The meaning of prefix `&` is not quite the same as in C. In particular, any changes to the referenced
variables will not be visible in Julia unless the type is mutable (declared via `mutable struct`). However,
even for immutable structs it will not cause any harm for called functions to attempt such modifications
(that is, writing through the passed pointers). Moreover, `&` may be used with any expression,
such as `&0` or `&f(x)`.

When a scalar value is passed with `&` as an argument of type `Ptr{T}`, the value will first be
converted to type `T`.

## Some Examples of C Wrappers
## C Wrapper Examples

Here is a simple example of a C wrapper that returns a `Ptr` type:

Expand Down Expand Up @@ -855,6 +819,25 @@ external C function. As a result, the code may produce a memory leak if `result_
freed by the garbage collector, or if the garbage collector prematurely frees `result_array`,
the C function may end up throwing an invalid memory access exception.

## Fortran Wrapper Example

The following example computes a dot product using a Fortran BLAS function. Note the use
of `Ref` and `Ptr`.

```julia
function compute_dot(DX::Vector{Float64}, DY::Vector{Float64})
@assert length(DX) == length(DY)
n = length(DX)
incx = incy = 1
product = ccall((:ddot_, "libLAPACK"),
Float64,
(Ref{Int32}, Ptr{Float64}, Ref{Int32}, Ptr{Float64}, Ref{Int32}),
n, DX, incx, DY, incy)
return product
end
```


## Garbage Collection Safety

When passing data to a [`ccall`](@ref), it is best to avoid using the [`pointer`](@ref) function.
Expand Down Expand Up @@ -1079,3 +1062,31 @@ For more details on how to pass callbacks to C libraries, see this [blog post](h

For direct C++ interfacing, see the [Cxx](https://github.com/Keno/Cxx.jl) package. For tools to create C++
bindings, see the [CxxWrap](https://github.com/JuliaInterop/CxxWrap.jl) package.



[^1]: Non-library function calls in both C and Julia can be inlined and thus may have
even less overhead than calls to shared library functions. When both libraries and executables are
generated by LLVM, it is possible to perform whole-program optimizations that can even optimize
across this boundary, but Julia does not yet support that. In the future, however, it may do so,
yielding even greater performance gains.

[^2]: The [Clang package](https://github.com/ihnorton/Clang.jl) can be used to auto-generate Julia code
from a C header file.

[^3]: For the conversion of ccall parameters, the actual conversion would look something like this:

```julia
ccall((:foo, "libfoo"), Cvoid, (Int32, Float64),
Base.unsafe_convert(Int32, Base.cconvert(Int32, x)),
Base.unsafe_convert(Float64, Base.cconvert(Float64, y)))
```

[`Base.cconvert`](@ref) normally just calls [`convert`](@ref), but can be defined to return an
arbitrary new object more appropriate for passing to C.
This should be used to perform all allocations of memory that will be accessed by the C code.
For example, this is used to convert an `Array` of objects (e.g. strings) to an array of pointers.

[`Base.unsafe_convert`](@ref) handles conversion to [`Ptr`](@ref) types. It is considered unsafe because
converting an object to a native pointer can hide the object from the garbage collector, causing
it to be freed prematurely.

0 comments on commit f7d3abe

Please sign in to comment.