Support for Shake256 extendable output function

CHANGES.md

@@ -6,6 +6,8 @@
 
 - Added support for [Zstandard](https://github.com/facebook/zstd) compression as a much more efficient alternative to ZLIB.
 The performance improvement should be expected to be around 20-30x.
+- Added support for Shake256 extendable output function for pseudo-random number generation.
+The user can choose to set the default to Blake2xb (faster) or Shake256, when configuring the build system.
 
 ### API Changes
 
@@ -23,6 +25,9 @@ The `BigUInt` class was only used by the `IntegerEncoder`.
 These were poorly named and have been replaced with overloads of `KeyGenerator::create_relin_keys` and `KeyGenerator::create_galois_keys` that take an out-parameter of type `RelinKeys` or `GaloisKeys`.
 - Renamed `IntArray` to `DynArray` (dynamic array) and removed unnecessary limitations on the object type template parameter.
 - Added const overloads for `DynArray::begin` and `DynArray::end`.
+- Added a `Shake256PRNG` and `Shake256PRNGFactory` classes.
+Renamed `BlakePRNG` class to `Blake2xbPRNG`, and `BlakePRNGFactory` class to `Blake2xbPRNGFactory`.
+- Added a serializable `UniformRandomGeneratorInfo` class that represents the type of an extendable output function and a seed value.
 
 ### Other
 

CMakeLists.txt

@@ -432,16 +432,15 @@ if(MSVC AND BUILD_SHARED_LIBS)
 endif()
 
 # [option] SEAL_THROW_ON_TRANSPARENT_CIPHERTEXT (default: ON)
-# Throw an exception if the second polynomial of a ciphertext is zero.
-set(SEAL_THROW_ON_TRANSPARENT_CIPHERTEXT_STR "Throw an exception when a member of Evaluator outputs a transparent ciphertext")
+set(SEAL_THROW_ON_TRANSPARENT_CIPHERTEXT_STR "Throw an exception when Evaluator outputs a transparent ciphertext")
 option(SEAL_THROW_ON_TRANSPARENT_CIPHERTEXT ${SEAL_THROW_ON_TRANSPARENT_CIPHERTEXT_STR} ON)
 mark_as_advanced(FORCE SEAL_THROW_ON_TRANSPARENT_CIPHERTEXT)
 
-# [option] SEAL_USE_GAUSSIAN (default: OFF)
-# Use Gaussian distribution for error if set to ON, use centered binomial otherwise.
-set(SEAL_USE_GAUSSIAN_STR "Use Gaussian for error distribution instead of centerd binomial")
-option(SEAL_USE_GAUSSIAN ${SEAL_USE_GAUSSIAN_STR} OFF)
-mark_as_advanced(FORCE SEAL_USE_GAUSSIAN)
+# [option] SEAL_USE_GAUSSIAN_NOISE (default: OFF)
+# Use Gaussian distribution for noise sampling if set to ON, use centered binomial otherwise.
+set(SEAL_USE_GAUSSIAN_NOISE_STR "Use a rounded Gaussian distribution for noise sampling instead of a Centered Binomial Distribution")
+option(SEAL_USE_GAUSSIAN_NOISE ${SEAL_USE_GAUSSIAN_NOISE_STR} OFF)
+mark_as_advanced(FORCE SEAL_USE_GAUSSIAN_NOISE)
 
 # [option] SEAL_USE_INTRIN (default: ON)
 set(SEAL_USE_INTRIN_OPTION_STR "Use intrinsics")

NOTICE

@@ -1,2 +1,5 @@
-This software uses the BLAKE2 library (https://github.com/BLAKE2/BLAKE2)
+This software includes parts of the BLAKE2 library (https://github.com/BLAKE2/BLAKE2).
 The BLAKE2 library is licensed under CC0 Universal, version 1.0. You can find a copy of this license at https://creativecommons.org/publicdomain/zero/1.0/legalcode
+
+This software includes parts of the Kyber library (https://github.com/pq-crystals/kyber).
+The Kyber library is licensed under CC0 Universal, version 1.0. You can find a copy of this license at https://creativecommons.org/publicdomain/zero/1.0/legalcode

dotnet/examples/6_Serialization.cs

@@ -81,10 +81,6 @@ that output Serializable<T> objects also have overloads that take a normal
             can be convenient for local testing where no serialization is needed and the
             object needs to be used at the point of construction. Such an object can no
             longer be transformed back to a seeded state.
-
-            Due to the internal representation of Microsoft SEAL objects, the size saving
-            from using seeded objects only has an effect when a compressed serialization
-            method is used, as we will demonstrate in the example below.
             */
 
             /*
@@ -146,15 +142,27 @@ providing EncryptionParameters.Save with the desired compression mode as
                     long size = parms.Save(sharedStream, ComprModeType.ZSTD);
 
                 If Microsoft SEAL is compiled with Zstandard or ZLIB support, the default
-                is to use one of them. If available, Zstandard is preferred over ZLIB.
+                is to use one of them. If available, Zstandard is preferred over ZLIB due
+                to its speed.
+
+                Compression can have a substantial impact on the serialized data size,
+                because ciphertext and key data consists of many uniformly random integers
+                modulo the CoeffModulus primes. Especially when using CKKS, the primes in
+                CoeffModulus can be relatively small compared to the 64-bit words used to
+                store the ciphertext and key data internally. Serialization writes full
+                64-bit words to the destination buffer or stream, possibly leaving in many
+                zero bytes corresponding to the high-order bytes of the 64-bit words. One
+                convenient way to get rid of these zeros is to apply a general-purpose
+                compression algorithm on the encrypted data. The compression rate can be
+                significant (up to 50-60%) when using CKKS with small primes.
                 */
 
                 /*
-                In many cases, when working with fixed size memory, it is necessary
-                to know ahead of time an upper bound on the serialized data size to
-                allocate enough memory. This information is returned by the
-                EncryptionParameters.SaveSize function. This function accepts the
-                desired compression mode, or uses the default option otherwise.
+                In many cases, when working with fixed size memory, it is necessary to know
+                ahead of time an upper bound on the serialized data size to allocate enough
+                memory. This information is returned by the EncryptionParameters.SaveSize
+                function. This function accepts the desired compression mode, or uses the
+                default option otherwise.
 
                 In more detail, the output of EncryptionParameters.SaveSize is as follows:
 
@@ -246,7 +254,9 @@ absolutely want to create as seeded objects to minimize communication
 
                 /*
                 We serialize both relinearization keys to demonstrate the concrete size
-                difference.
+                difference. If compressed serialization is used, the compression rate
+                will be the same in both cases. We omit specifying the compression mode
+                to use the default, as determined by the Microsoft SEAL build system.
                 */
                 long sizeRlk = rlk.Save(dataStream);
                 long sizeRlkBig = rlkBig.Save(dataStream);
@@ -318,10 +328,6 @@ We would also like to draw attention to the fact there we could easily
                 serialize multiple Microsoft SEAL objects sequentially in a stream. Each
                 object writes its own size into the stream, so deserialization knows
                 exactly how many bytes to read. We will see this working below.
-
-                Finally, we would like to emphasize the point that none of these methods
-                provide any space savings when ComprModeType.None is used, e.g., when
-                Microsoft SEAL is not compiled with support for ZLIB or Zstandard.
                 */
             }
 

native/examples/6_serialization.cpp

@@ -74,10 +74,6 @@ void example_serialization()
     can be convenient for local testing where no serialization is needed and the
     object needs to be used at the point of construction. Such an object can no
     longer be transformed back to a seeded state.
-
-    Due to the internal representation of Microsoft SEAL objects, the size saving
-    from using seeded objects only has an effect when a compressed serialization
-    method is used, as we will demonstrate in the example below.
     */
 
     /*
@@ -142,7 +138,19 @@ void example_serialization()
             auto size = parms.save(shared_stream, compr_mode_type::zstd);
 
         If Microsoft SEAL is compiled with Zstandard or ZLIB support, the default
-        is to use one of them. If available, Zstandard is preferred over ZLIB.
+        is to use one of them. If available, Zstandard is preferred over ZLIB due
+        to its speed.
+
+        Compression can have a substantial impact on the serialized data size,
+        because ciphertext and key data consists of many uniformly random integers
+        modulo the coeff_modulus primes. Especially when using CKKS, the primes in
+        coeff_modulus can be relatively small compared to the 64-bit words used to
+        store the ciphertext and key data internally. Serialization writes full
+        64-bit words to the destination buffer or stream, possibly leaving in many
+        zero bytes corresponding to the high-order bytes of the 64-bit words. One
+        convenient way to get rid of these zeros is to apply a general-purpose
+        compression algorithm on the encrypted data. The compression rate can be
+        significant (up to 50-60%) when using CKKS with small primes.
         */
 
         /*
@@ -252,7 +260,9 @@ void example_serialization()
 
         /*
         We serialize both relinearization keys to demonstrate the concrete size
-        difference.
+        difference. If compressed serialization is used, the compression rate
+        will be the same in both cases. We omit specifying the compression mode
+        to use the default, as determined by the Microsoft SEAL build system.
         */
         auto size_rlk = rlk.save(data_stream);
         auto size_rlk_big = rlk_big.save(data_stream);
@@ -283,7 +293,7 @@ void example_serialization()
         The client will not compute on ciphertexts that it creates, so it can
         just as well create Serializable<Ciphertext> objects. In fact, we do
         not even need to name those objects and instead immediately call
-        Serializable<Ciphertext>.save.
+        Serializable<Ciphertext>::save.
         */
         auto size_encrypted1 = encryptor.encrypt(plain1).save(data_stream);
 
@@ -317,10 +327,6 @@ void example_serialization()
         serialize multiple Microsoft SEAL objects sequentially in a stream. Each
         object writes its own size into the stream, so deserialization knows
         exactly how many bytes to read. We will see this working below.
-
-        Finally, we would like to emphasize the point that none of these methods
-        provide any space savings when compr_mode_type::none is used, e.g., when
-        Microsoft SEAL is not compiled with support for ZLIB or Zstandard.
         */
     }
 

native/src/seal/ciphertext.cpp

@@ -2,7 +2,6 @@
 // Licensed under the MIT license.
 
 #include "seal/ciphertext.h"
-#include "seal/randomgen.h"
 #include "seal/util/defines.h"
 #include "seal/util/pointer.h"
 #include "seal/util/polyarithsmallmod.h"
@@ -115,13 +114,17 @@ namespace seal
         coeff_modulus_size_ = coeff_modulus_size;
     }
 
-    void Ciphertext::expand_seed(const SEALContext &context, const random_seed_type &seed)
+    void Ciphertext::expand_seed(const SEALContext &context, const UniformRandomGeneratorInfo &prng_info)
     {
         auto context_data_ptr = context.get_context_data(parms_id_);
 
-        // Set up the BlakePRNG with a given seed.
-        // Rejection sampling to generate a uniform random polynomial.
-        sample_poly_uniform(make_shared<BlakePRNG>(seed), context_data_ptr->parms(), data(1));
+        // Set up a PRNG from the given info and sample the second polynomial
+        auto prng = prng_info.make_prng();
+        if (!prng)
+        {
+            throw logic_error("unsupported prng_type");
+        }
+        sample_poly_uniform(prng, context_data_ptr->parms(), data(1));
     }
 
     streamoff Ciphertext::save_size(compr_mode_type compr_mode) const
@@ -138,8 +141,8 @@ namespace seal
                 data_.pool());
 
             data_size = add_safe(
-                safe_cast<size_t>(alias_data.save_size(compr_mode_type::none)), // data_(0)
-                sizeof(random_seed_type));                                      // seed
+                safe_cast<size_t>(alias_data.save_size(compr_mode_type::none)),                    // data_(0)
+                static_cast<size_t>(UniformRandomGeneratorInfo::SaveSize(compr_mode_type::none))); // seed
         }
         else
         {
@@ -180,8 +183,9 @@ namespace seal
 
             if (has_seed_marker())
             {
-                random_seed_type seed;
-                copy_n(data(1) + 1, seed.size(), seed.begin());
+                UniformRandomGeneratorInfo info;
+                size_t info_size = static_cast<size_t>(UniformRandomGeneratorInfo::SaveSize(compr_mode_type::none));
+                info.load(reinterpret_cast<const seal_byte *>(data(1) + 1), info_size);
 
                 size_t data_size = data_.size();
                 size_t half_size = data_size / 2;
@@ -195,8 +199,8 @@ namespace seal
                 swap(alias_data.data_, alias_ptr);
                 alias_data.save(stream, compr_mode_type::none);
 
-                // Save the seed
-                stream.write(reinterpret_cast<char *>(&seed), sizeof(random_seed_type));
+                // Save the UniformRandomGeneratorInfo
+                info.save(stream, compr_mode_type::none);
             }
             else
             {
@@ -285,15 +289,17 @@ namespace seal
             auto seeded_uint64_count = poly_modulus_degree64 * coeff_modulus_size64;
 
             // This is the case where we need to expand a seed, otherwise full
-            // ciphertext data was (possibly) loaded and do nothing
+            // ciphertext data was already (possibly) loaded and we are done
             if (unsigned_eq(new_data.data_.size(), seeded_uint64_count))
             {
-                // Single polynomial size data was loaded, so we are in the
-                // seeded ciphertext case. Next load the seed.
-                random_seed_type seed;
-                stream.read(reinterpret_cast<char *>(&seed), sizeof(random_seed_type));
+                // Single polynomial size data was loaded, so we are in the seeded
+                // ciphertext case. Next load the UniformRandomGeneratorInfo.
+                UniformRandomGeneratorInfo prng_info;
+                prng_info.load(stream);
+
+                // Set up a UniformRandomGenerator and expand
                 new_data.data_.resize(total_uint64_count);
-                new_data.expand_seed(context, seed);
+                new_data.expand_seed(context, prng_info);
             }
 
             // Verify that the buffer is correct

native/src/seal/ciphertext.h

@@ -668,7 +668,7 @@ namespace seal
 
         void resize_internal(std::size_t size, std::size_t poly_modulus_degree, std::size_t coeff_modulus_size);
 
-        void expand_seed(const SEALContext &context, const random_seed_type &seed);
+        void expand_seed(const SEALContext &context, const UniformRandomGeneratorInfo &prng_info);
 
         void save_members(std::ostream &stream) const;
 

native/src/seal/encryptionparams.h

@@ -386,8 +386,7 @@ namespace seal
 
         @param[in] stream The stream to load the EncryptionParameters from
         @throws std::logic_error if the data cannot be loaded by this version of
-        Microsoft SEAL, if the loaded data is invalid, if decompression failed,
-        or if the loaded size exceeds in_size_bound
+        Microsoft SEAL, if the loaded data is invalid or if decompression failed
         @throws std::runtime_error if I/O operations failed
         */
         inline std::streamoff load(std::istream &stream)
@@ -430,8 +429,7 @@ namespace seal
         @throws std::invalid_argument if in is null or if size is too small to
         contain a SEALHeader
         @throws std::logic_error if the data cannot be loaded by this version of
-        Microsoft SEAL, if the loaded data is invalid, if decompression failed,
-        or if the loaded size exceeds in_size_bound
+        Microsoft SEAL, if the loaded data is invalid, or if decompression failed
         @throws std::runtime_error if I/O operations failed
         */
         inline std::streamoff load(const seal_byte *in, std::size_t size)

native/src/seal/encryptor.h

@@ -415,8 +415,6 @@ namespace seal
 
         Encryptor &operator=(Encryptor &&assign) = delete;
 
-        MemoryPoolHandle pool_ = MemoryManager::GetPool(mm_prof_opt::FORCE_NEW, true);
-
         void encrypt_zero_internal(
             parms_id_type parms_id, bool is_asymmetric, bool save_seed, Ciphertext &destination,
             MemoryPoolHandle pool = MemoryManager::GetPool()) const;

native/src/seal/modulus.h

@@ -169,7 +169,7 @@ namespace seal
         */
         SEAL_NODISCARD inline bool operator!=(const Modulus &compare) const noexcept
         {
-            return !(value_ == compare.value_);
+            return !operator==(compare);
         }
 
         /**
@@ -179,7 +179,7 @@ namespace seal
         */
         SEAL_NODISCARD inline bool operator!=(std::uint64_t compare) const noexcept
         {
-            return value_ != compare;
+            return !operator==(compare);
         }
 
         /**