Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-3473

OpenSSL 1.1.1h causes a segfault in RSA_generate_key_ex() on Solaris 10 x86_64



    • Type: Task
    • Status: Open
    • Priority: (None)
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Packaging
    • Labels:


      Starting test: hash_test.c
      [New Thread 1 (LWP 1)]
      Program received signal SIGSEGV, Segmentation fault.
      [Switching to Thread 1 (LWP 1)]
      ctr_BCC_init (ctr=0x808adbc) at crypto/rand/drbg_ctr.c:110
      110     crypto/rand/drbg_ctr.c: No such file or directory.
      #0  ctr_BCC_init (ctr=0x808adbc) at crypto/rand/drbg_ctr.c:110
      #1  ctr_df (in3len=<optimized out>, in3=<optimized out>, in2len=<optimized out>, in2=<optimized out>, in1len=<optimized out>, 
          in1=0x808b390 "\224\334\323\306\241\312\317S| p|\372\234\234\212{\201\026\340\064\306D\307\265\262\227\374/\245\"\005", ctr=0x808adbc) at crypto/rand/drbg_ctr.c:171
      #2  ctr_update (drbg=drbg@entry=0x808ad58, in1=in1@entry=0x808b390 "\224\334\323\306\241\312\317S| p|\372\234\234\212{\201\026\340\064\306D\307\265\262\227\374/\245\"\005", 
          in1len=in1len@entry=32, in2=0xfedd2be0 <ossl_pers_string> "OpenSSL NIST SP 800-90A DRBG", in2len=28, nonce=0x808b3c8 "\267", noncelen=24) at crypto/rand/drbg_ctr.c:254
      #3  0xfed4869b in drbg_ctr_instantiate (drbg=<optimized out>, entropy=<optimized out>, entropylen=<optimized out>, nonce=<optimized out>, noncelen=<optimized out>, pers=<optimized out>, 
          perslen=<optimized out>) at crypto/rand/drbg_ctr.c:286
      #4  0xfed48d57 in RAND_DRBG_instantiate (drbg=0x808ad58, pers=0xfedd2be0 <ossl_pers_string> "OpenSSL NIST SP 800-90A DRBG", perslen=28) at crypto/rand/drbg_lib.c:348
      #5  0xfed49b9d in drbg_setup (parent=0x0) at crypto/rand/drbg_lib.c:891
      #6  do_rand_drbg_init () at crypto/rand/drbg_lib.c:920
      #7  do_rand_drbg_init_ossl_ () at crypto/rand/drbg_lib.c:905
      #8  0xfe858cf5 in pthread_once () from /lib/libc.so.1
      #9  0xfed715dd in CRYPTO_THREAD_run_once (once=0xfee1d700 <rand_drbg_init>, init=0xfed49ad0 <do_rand_drbg_init_ossl_>) at crypto/threads_pthread.c:118
      #10 0xfed4a177 in RAND_DRBG_get0_private () at crypto/rand/drbg_lib.c:1133
      #11 0xfed4b4f5 in RAND_priv_bytes (buf=0x808ac80 "", num=64) at crypto/rand/rand_lib.c:925
      #12 0xfec98e34 in bnrand (flag=PRIVATE, bottom=1, top=1, bits=512, rnd=0x8088690) at crypto/bn/bn_rand.c:46
      #13 BN_priv_rand (rnd=0x8088690, bits=512, top=1, bottom=1) at crypto/bn/bn_rand.c:109
      #14 0xfec9759f in probable_prime (mods=<optimized out>, safe=<optimized out>, bits=<optimized out>, rnd=<optimized out>) at crypto/bn/bn_prime.c:280
      #15 BN_generate_prime_ex (ret=0x8088690, bits=512, safe=0, add=0x0, rem=0x0, cb=0x0) at crypto/bn/bn_prime.c:92
      #16 0xfed52add in rsa_builtin_keygen (cb=0x0, e_value=0x8088610, primes=2, bits=1024, rsa=<optimized out>) at crypto/rsa/rsa_gen.c:164
      #17 RSA_generate_multi_prime_key (rsa=0x80884b0, bits=1024, primes=2, e_value=0x8088610, cb=0x0) at crypto/rsa/rsa_gen.c:61
      #18 0xfed531ac in RSA_generate_key_ex (rsa=0x80884b0, bits=1024, e_value=0x8088610, cb=0x0) at crypto/rsa/rsa_gen.c:37
      #19 0x0805748c in tests_setup () at hash_test.c:57
      #20 0x080590af in main () at hash_test.c:271

      Seems to be likely caused by https://github.com/openssl/openssl/commit/53eb05bdf00d7237e3b12976c2ac38d68206eb13 which we can just revert on Solaris 10 x86_64, it's an optimization patch.




            • Assignee:
              Aleksei Aleksei Shpakovskii
              vpodzime Vratislav Podzimek
            • Votes:
              0 Vote for this issue
              1 Start watching this issue


              • Created:

                Summary Panel