There are two value types, and the choice between them is about where the scale lives.
[Decimal{P,S,T}](@ref Decimal) puts the scale in the type. It holds one
field, unscaled::T, and means unscaled * 10^-S with the invariant
|unscaled| < 10^P. P is the precision (total decimal digits), S the
scale (fractional digits, 0 <= S <= P), and T the storage integer.
A vector stores coefficients contiguously. Arrow buffers with the same width
and host byte order can share this layout. Parquet byte arrays use big-endian
two's-complement coefficients and require encoding. DuckDB uses 16/32/64/128-bit
storage for precisions up to 4/9/18/38; it has no 256-bit decimal tier.
See the Arrow format,
Parquet decimal specification,
and DuckDB numeric types.
Four aliases name this package's storage tiers at their maximum precision:
| alias | expands to | digits | storage | typical use |
|---|---|---|---|---|
[DataDecimals.Decimal32{S}](@ref DataDecimals.Decimal32) |
Decimal{9,S,Int32} |
9 | 32-bit | narrow Arrow/Parquet columns |
[DataDecimals.Decimal64{S}](@ref DataDecimals.Decimal64) |
Decimal{18,S,Int64} |
18 | 64-bit | money, most SQL DECIMAL |
[DataDecimals.Decimal128{S}](@ref DataDecimals.Decimal128) |
Decimal{38,S,Int128} |
38 | 128-bit | DuckDB/Spark default width |
[DataDecimals.Decimal256{S}](@ref DataDecimals.Decimal256) |
Decimal{76,S,DataDecimals.Int256} |
76 | 256-bit | Arrow DataDecimals.Decimal256 |
Write Decimal{P,S} when a schema pins a precision narrower than its tier's
maximum: Decimal{7,2} accepts fewer digits than DataDecimals.Decimal32{2} before
throwing, though both store an Int32. The storage type is filled in from
P whenever you leave it off, so Decimal{9,2} is another spelling of
DataDecimals.Decimal32{2}.
[DataDecimals.DecimalValue{T}](@ref DataDecimals.DecimalValue) is the runtime-scale sibling: it
stores the scale (an Int32 in 0:16383) alongside the coefficient. Use it
when the scale travels with each value rather than with the column, as with a
PostgreSQL numeric's per-value dscale, a value whose scale is not known
until run time, or the result of DataDecimals.normalize. It is still
isbits, but it is a word wider and its arithmetic has to align scales at run
time, so Decimal{P,S} is the better default for columnar data.
Both are subtypes of DataDecimals.AbstractDecimal <: Real. There is no
AbstractFloat subtyping, and there are no NaN or Inf values: isnan is
always false, isfinite always true, and an operation that would produce
a non-value throws.
dec"1.25" # literal, minimal fitting type
dec"1_000_000.25" # underscores allowed between digit groups
dec"1.5e-3" # exponents fold into the scale
DataDecimals.Decimal64{2}("1234.56") # from a string (exact core scanner)
using Parsers # parse/tryparse come from the Parsers extension
parse(DataDecimals.Decimal64{2}, "1234.567") # 1234.57, rounds half-even at the target scale
tryparse(DataDecimals.Decimal64{2}, "nope") # nothing
DataDecimals.Decimal64{2}(5) # from an Integer: exact or throws
DataDecimals.Decimal64{2}(1//4) # from a Rational: exact or throws
DataDecimals.Decimal64{2}(0.1) # from a Float: rounds half-even at scale 2
reinterpret(DataDecimals.Decimal64{2}, 123456) # from an unscaled coefficient, uncheckedThe rule is exact-or-throw, with two documented exceptions. Construction from
an AbstractFloat rounds half-even at the target scale, as cross-representation
float conversions in Base do; a DataDecimals.DecimalValue built from a float instead keeps
the exact binary expansion, which is what a runtime scale is for:
DataDecimals.Decimal64{2}(0.1) # 0.10
DataDecimals.DecimalValue(0.1) # 0.1000000000000000055511151231257827021181583404541015625String construction and parse also round half-even at the target scale.
convert(D, x) follows the numeric constructor: float sources round, while
integer, rational, and decimal sources are exact or throw. Use an exact string
literal or rational source when binary float rounding is not intended.
@dec_str produces the minimal type that fits the literal, so
dec"1.25" is a Decimal{3,2,Int32} and not a money type. That suits a
constant in an expression, where it promotes to whatever the other operand
needs; annotate the type explicitly for a column.
reinterpret is the zero-copy path: it builds a value directly from an
unscaled coefficient without checking the |u| < 10^P invariant, because in
a wire decode the invariant is the producer's contract and re-checking it per
element is the cost being avoided.
+ and - promote both operands value-preservingly and then compute
checked: a result that no longer fits throws OverflowError. They never wrap
and never silently round.
DataDecimals.Decimal64{2}("1.20") + DataDecimals.Decimal64{2}("0.05") # 1.25 :: Decimal{18,2,Int64}
typemax(DataDecimals.Decimal64{2}) + DataDecimals.Decimal64{2}("1.00") # OverflowError* is exact: the result has scale S1 + S2 and precision min(P1 + P2, 76).
Multiplying by a plain Integer is scale-preserving instead, because
multiplying by a count should not widen the scale.
DataDecimals.Decimal64{2}("1.25") * DataDecimals.Decimal64{3}("2.500") # 3.12500 :: Decimal{36,5,Int128}
DataDecimals.Decimal64{2}("2.50") * 3 # 7.50 :: Decimal{37,2,Int128}/ rounds half-even at scale max(S1, S2). DataDecimals.divide is the same
operation with the rounding mode spelled out:
DataDecimals.Decimal64{2}("1.00") / DataDecimals.Decimal64{2}("3.00") # 0.33
DataDecimals.divide(DataDecimals.Decimal64{2}("1.00"), DataDecimals.Decimal64{2}("3.00"), RoundUp) # 0.34sum widens to the Int128 tier, where overflow is provably impossible
below 1.7e20 elements, so the accumulation loop needs no per-element check
and vectorizes while staying safe:
sum(fill(DataDecimals.Decimal64{2}("0.10"), 10)) # 1.00 :: Decimal{38,2,Int128}div, rem, mod, fld, cld, and divrem are real implementations
rather than float fallbacks, and take a rounding mode where Base's do.
Elementwise a .+ b, a .- b, and a .* b over same-shaped vectors hit SIMD
kernels that compute per lane and check overflow once for the whole array.
There is no global rounding state in the package. Every rounding decision is
an argument at a call site, and the seven modes accepted are RoundNearest
(half-even, the default), RoundNearestTiesAway, RoundNearestTiesUp,
RoundToZero, RoundFromZero, RoundDown, and RoundUp. An eighth,
DataDecimals.RoundExact, requires exactness: it throws
InexactError instead of dropping a digit.
# change the scale of a value
DataDecimals.rescale(DataDecimals.Decimal64{2}, DataDecimals.Decimal64{4}("1.2356"), RoundUp) # 1.24
DataDecimals.rescale(DataDecimals.DecimalValue(12345, 3), 1) # 12.3
round(DataDecimals.Decimal64{2}, 1.005) # 1.00 (same thing, Base spelling)
# round within a type
round(DataDecimals.Decimal64{4}("1.2346"); digits=2) # 1.2300
trunc(DataDecimals.Decimal64{2}("1.99")) # 1.00
floor(DataDecimals.Decimal64{2}("-1.01")) # -2.00
ceil(DataDecimals.Decimal64{2}("1.01")) # 2.00
# round out to an integer
round(Int, DataDecimals.Decimal64{2}("1.50")) # 2
floor(Int, DataDecimals.Decimal64{2}("1.99")) # 1round(x; digits) stays exact and in-type; Base's generic fallback for a
custom Real returns a Float64, and that is overridden here. There is no
sigdigits keyword, since a fixed-point type has no floating significand.
convert between decimal types is the exact-or-throw counterpart of
DataDecimals.rescale; reach for DataDecimals.rescale when the intent is "round this into
that scale".
Equality and ordering are by numeric value, across scales, precisions,
storage types, and other Real types. The representation still remembers its
scale, so string prints the trailing zeros, but the number does not:
dec"1.20" == dec"1.2" == 12//10 # true
hash(DataDecimals.Decimal64{2}("1.25")) == hash(1.25) # true
sort([dec"3.5", dec"1.25", dec"2.0"]) # works; sorts on the coefficientHashing goes through Base.decompose, so a decimal hashes identically to an
equal Int, Rational, or Float64, which is what makes Dict keys,
unique, and DataFrames groupby/join behave.
Between decimals, ≈ is exact equality, because Base's rtoldefault is zero
for a non-AbstractFloat Real. That is the same behaviour Rational has,
and it is arguably what money wants; pass rtol or atol for a tolerance.
Comparing against a Float64 brings that type's default tolerance with it,
again as for Rational.
string and print always produce the plain positional form, which is a
valid SQL literal, CSV field, and JSON number, and never scientific notation:
string(DataDecimals.Decimal64{2}("-12.30")) # "-12.30"show, and therefore repr, gives the round-trippable typed form,
Decimal{18,2,Int64}("-12.30"). At the REPL, though, values display through
the text/plain method, which prints plain digits and only switches to
scientific notation once the positional form passes 44 characters:
julia> DataDecimals.Decimal64{2}("1234.56")
1234.56
julia> DataDecimals.DecimalValue{DataDecimals.Int256}(123, 50)
1.23E-48parse and tryparse, defined by the Parsers extension, accept an optional
sign, digits, an optional decimal point, and an optional e/E exponent,
with surrounding ASCII whitespace allowed. They round half-even into the
target scale. The string constructors (DataDecimals.Decimal64{2}("1.25")) and dec"..."
literals use the package's own exact scanner and need no extension.
Four public names cover byte-level interop:
using DataDecimals: unscaled, scale, writedecimal!, decimallength
x = DataDecimals.Decimal64{2}("-1.25")
unscaled(x) # -125 :: Int64, what the buffer stores
scale(x) # 2
reinterpret(typeof(x), unscaled(x)) === x # true
decimallength(x) # 5, bytes needed, computed without formatting
buf = zeros(UInt8, 16)
pos = writedecimal!(buf, 1, x) # 6, the position after the written bytes
String(buf[1:pos-1]) # "-1.25"[writedecimal!](@ref DataDecimals.writedecimal!) is the allocation-free
formatter that string is built on; it bounds-checks nothing, so reserve
[decimallength(x)](@ref DataDecimals.decimallength) bytes first. Together with
reinterpret, this is enough to read and write decimal columns without
materialising an intermediate String.
DataDecimals.normalize is the other representation-level tool: it
strips trailing zeros from the coefficient and lowers the scale to match,
returning a DataDecimals.DecimalValue, since the scale is then a property of the value
rather than the type. The number is unchanged; only the digits a wire format
would carry are.
!!! note
normalize is public but not exported, so that it does not collide with
LinearAlgebra.normalize; call it as DataDecimals.normalize.
These load automatically as package extensions when the companion package is present.
Byte-level parse, tryparse, and parsenext over strings or byte spans,
with decimal= and rounding= keywords. This is the entry point for a CSV
reader or wire decoder.
using DataDecimals, Parsers
Parsers.parse(DataDecimals.Decimal64{2}, "1234.56") # 1234.56
Parsers.parse(DataDecimals.Decimal64{2}, "1,25"; decimal=',') # 1.25
Parsers.parse(DataDecimals.Decimal64{2}, "1.005"; rounding=RoundUp) # 1.01
Parsers.tryparse(DataDecimals.Decimal64{2}, "nope") # nothing
Parsers.tryparse(DataDecimals.Decimal64{2}, "1.005"; rounding=DataDecimals.RoundExact) # nothing
buf = codeunits("12.34,56.78")
Parsers.parsenext(DataDecimals.Decimal64{2}, buf, 1, length(buf)) # (12.34, 6, Parsers.RC_OK)parsenext returns (value, nextpos, returncode) and is the tokenizer entry
point: it stops at the first byte that cannot continue a decimal, so the
caller can step through a delimited line. It requires Parsers 3. Under
Parsers 2 the extension loads as a no-op, so that an environment which also
contains a Parsers-2-pinned package still resolves, and parse/tryparse
are unavailable until Parsers 3 is (string constructors and literals still
work). The extension also defines Base.parse/Base.tryparse for decimal
types, so plain parse(DataDecimals.Decimal64{2}, s) is the same code path once Parsers
is loaded.
%f, %e, and %g round the decimal value half-even at the requested
precision before conversion to BigFloat. Binary precision grows with the
requested output precision. This preserves decimal ties and long output. %d is exact-or-InexactError, matching Rational.
@printf("%.2f", DataDecimals.Decimal64{2}("1234.56")) # 1234.56
@printf("%d", DataDecimals.Decimal64{2}("1234.00")) # 1234
@printf("%d", DataDecimals.Decimal64{2}("1234.56")) # InexactErrorDataDecimals serialize as JSON numbers carrying their exact digits, neither as
strings nor Float64-rounded (the default path for an unknown Real
stringifies convert(Float64, x)). Requires JSON 1.x.
JSON.json((price = DataDecimals.Decimal64{2}("1234.56"), qty = 3))
# {"price":1234.56,"qty":3}A*B, dot, tr, A+B, and kron stay exact decimal, widening the result
type as the product requires. Factorizations, namely lu, cholesky, qr,
svd, eigen, hessenberg, schur, lq, and the functions built on them
(det, logdet, inv, eigvals, svdvals, cond, nullspace, pinv,
rank), convert to Float64 first. See LinearAlgebra under
Semantics for why. If you need an exact factorization, lu(Rational.(A))
gives one.
Only Decimal, Decimal64, and @dec_str are exported. Other public names use the
DataDecimals. namespace. Explicit imports such as using DataDecimals: DecimalValue
are supported. sign(x) returns the storage integer's sign, so it works even
when the decimal type cannot hold 1. Mixing a runtime-scale DecimalValue
with a rational promotes to Rational{BigInt} so deep scales stay exact.
The fixed family supports 1 <= P <= 76 and 0 <= S <= P. Arrow schemas can
carry negative scales or scales above their precision; those schemas need an
explicit adapter policy and cannot be mapped directly to Decimal{P,S}.
DecimalValue permits scales through 16383, but its coefficient is still
bounded by its storage integer. It does not cover all PostgreSQL numeric
values, including NaN, infinities, or coefficients beyond 256 bits.
Validate schema parameters, coefficients, byte order, and null masks before
using reinterpret for external data. Null values belong in missing or a
format's validity bitmap. A scalar reinterpret(D, u) is unchecked and can
truncate a wider integer. Use convert(D, x) to convert a numeric value.
The LinearAlgebra extension supports ordinary dense matrices, pivot arguments,
and diagonal/triangular inverses and solves with vector or dense matrix right
sides. Other structured or in-place operations can still use fixed-scale
arithmetic; convert inputs with float first. Use Rational{BigInt} inputs
when exact algebra is required.