Tuesday, April 7, 2015

build openssl 1.0.* on windows

problem

It's not trivial task - build OpenSSL library - especially if you are doing it first time.
By the way - OpenSSL has good documentation (and about building process on windows) - but nobody reads documentation.
So, here are my notes of that - mostly for myself, but also it can be useful for people, who find it in google.
It works on openssl 1.0.* versions. 1.1.* builds another way.


tools & versions

Will build OpenSSL by two compilers: Visual Studio 2013 and mingw.
Mingw-compiled version can be useful for binding with perl, for example.

  • Openssl source code. For today, last version of openssl - openssl-1.0.2a. Will build it. Download it. [link]
  • For Visual Studio compiling need perl - I'm using ActivePerl (v5.20.1.2000 x64). Download and install it. [link]
  • Optional - for compilation with Visual Studio for x86 platform Netwide Assembler - nasm (I have v2.11.08). Download and install it - after intalling add path to nasm.exe in %PATH%. [link]
  • For compiling with mingw you need msys (I have MSYS-20111123). Download and unpack it. [link]

lyrics

asm
In sources present assembler sources for x32 code. It makes x2-x4 better performance - link.
Earlier you could use masm, but as I see - from 1999 year it slowly disappears and now documentation says only nasm supported. Anyway - it's only for dying 32-bit mode.
If you don't want to install masm or you don't care about x2-x4 performance, you can compile it with 'no-asm' option (will be shown further) - in this case will be used C routines.

compilation time
Compilation with Visual Studio takes ~2 min.
Compilation with mingw takes ~16 min.

symbols
If you just build with visual studio by default - when you will use this library in your project, visual studio will print tons of annoying messages, like:
    1>libeay32.lib(cryptlib.obj) : warning LNK4099: PDB 'lib.pdb' was not found with 'libeay32.lib(cryptlib.obj)' or at 'D:\blablabla\Debug\lib.pdb'; linking object as if no debug info
so, if you also don't like this stuff - you have 2 paths to solve this problem:
  • embed symbols into lib files
  • copy symbols manually
embedding symbols
To embed symbols - after you generated nt.mak file (by command 'ms\do_ms', 'ms\do_nasm' or 'ms\do_win64a') but before you called nmake (described further) - open file 'ms\nt.mak' in any text editor and replace all (two) '/Zi' occurrences to '/Z7'.

copying symbols
After building (for example, you built in 'C:\mydir' directory), .pdb symbols will located into paths.
C:\mydir\out32if you build release static library
C:\mydir\out32dllif you build release dynamic library
C:\mydir\out32.dbgif you build debug static library
C:\mydir\out32dll.dbgif you build debug dynamic library
so, you can just copy symbols from this dir.

safeseh
And good news of SAFESEH - earlier on x86 you needed also fix 'ms\nt.mak' to use safeseh mechanism
(it makes your code more secure and get rid you from another annoying message from visual studio):
    1>libeay32.lib(x86cpuid.obj) : error LNK2026: module unsafe for SAFESEH image.
(http://eran.geek.co.il/wp/archives/3897 wrote about that)
but at the last version this warning disappeared (it seems this warning appeared on modules, compiled by masm). Well, anyway if you will get such warning - you know what to fix to get rid of it.

general principe of building in visual studio
First of all - you need to open visual studio command prompt.
as described here, for example - http://stackoverflow.com/questions/21476588/where-is-developer-command-prompt-for-vs2013
in "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\Shortcuts" shortcuts - "VS2013 x86 Native Tools Command Prompt" & "VS2013 x64 Native Tools Command Prompt"
You need open version, according to target platform.

Building process consists of 4 steps (and one optional step):
  1. First of all you need call 'Configure' perl script. In parameters you should pass:
    • target platform & build type:
      • VC-WIN32            release x86
      • debug-VC-WIN32        debug x86
      • VC-WIN64A            release x86-64
      • debug-VC-WIN64A        debug x86-64
    • on x86 platform if you don't want use asm, you should pass 'no-asm'
    • --prefix - it's destination path for builded files
    For example: > perl Configure VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_release_dynamic_noasm
  2. Second step - you need to generate nt.mak file. For that you need to exec one of next scripts:
    • > ms\do_ms            for 'no-asm' on x86
    • > ms\do_nasm        for x86 with asm
    • > ms\do_win64a        for x86-64 target platform
    Here are optional step - you can fix generated nt.mak file to add symbols (described above)
  3. Third step - build. Here are two options:
    • > nmake -f ms\ntdll.mak        to get dynamic libraries
    • > nmake -f ms\nt.mak        to get static libraries
  4. The last step - you should repeat previous line + add ' install' in the end of that. For example:
    • > nmake -f ms\nt.mak install
    This step just copied files into dest directory (which you entered in '--prefix' parameter)
Also I don't know how to make 'clear' on this building system - so, I've just removed everything in c:\mydif & unpacked here source files again. If you leave files from previous build with another parameters - it will lead to error during compilation.


action

Visual Studio 32-bit version

- unpack openssl-1.0.2a.tar.gz with any unpacker to C:\mydir (I'm using FAR manager for that)
- launch VS2013 x86 command prompt
> cd c:\mydir
then just choose what do you need, accordingly with general principles:

        ---------------------------------------------------
        # visual studio
        # x86
        # release
        # no-asm
        # dynamic
       
        > perl Configure VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_release_dynamic_noasm
        > ms\do_ms
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # release
        # no-asm
        # static
       
        > perl Configure VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_release_static_noasm
        > ms\do_ms
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # release
        # nasm
        # static
       
        > perl Configure VC-WIN32 --prefix=c:/_openssl-1.0.2a_x86_release_static_nasm
        > ms\do_nasm
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # release
        # nasm
        # dynamic
       
        > perl Configure VC-WIN32 --prefix=c:/_openssl-1.0.2a_x86_release_dynamic_nasm
        > ms\do_nasm
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------

        ---------------------------------------------------
        # visual studio
        # x86
        # debug
        # no-asm
        # dynamic
       
        > perl Configure debug-VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_debug_dynamic_noasm
        > ms\do_ms
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # debug
        # no-asm
        # static
       
        > perl Configure debug-VC-WIN32 no-asm --prefix=c:/_openssl-1.0.2a_x86_debug_static_noasm
        > ms\do_ms
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # debug
        # nasm
        # static
       
        > perl Configure debug-VC-WIN32 --prefix=c:/_openssl-1.0.2a_x86_debug_static_nasm
        > ms\do_nasm
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x86
        # debug
        # nasm
        # dynamic
       
        > perl Configure debug-VC-WIN32 --prefix=c:/_openssl-1.0.2a_x86_debug_dynamic_nasm
        > ms\do_nasm
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------

Visual Studio 64-bit version
- unpack openssl-1.0.2a.tar.gz with any unpacker to C:\mydir (I'm using FAR manager)
- launch VS2013 x64 command prompt (you can find it into 'start' menu)
> cd c:\mydir
then just choose what do you need, accordingly with general principles: 

        #---------------------------------------------------
        # visual studio
        # x64
        # release
        # static
       
        > perl Configure VC-WIN64A --prefix=C:\_openssl-1.0.2a_x64_release_static
        > ms\do_win64a
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------
        # visual studio
        # x64
        # release
        # dynamic
       
        > perl Configure VC-WIN64A --prefix=C:\_openssl-1.0.2a_x64_release_dynamic
        > ms\do_win64a
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------
        # visual studio
        # x64
        # debug
        # dynamic
       
        > perl Configure debug-VC-WIN64A --prefix=C:\_openssl-1.0.2a_x64_debug_dynamic
        > ms\do_win64a
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\ntdll.mak
        > nmake -f ms\ntdll.mak install
        ---------------------------------------------------
        # visual studio
        # x64
        # debug
        # static
       
        > perl Configure debug-VC-WIN64A --prefix=C:\_openssl-1.0.2a_x64_debug_static
        > ms\do_win64a
        // replace in ms\nt.mak all /Zi to /Z7 if you need - what for - described above
        > nmake -f ms\nt.mak
        > nmake -f ms\nt.mak install
        ---------------------------------------------------

mingw (here are considered only static mode)

Put openssl-1.0.2a.tar.gz to C:\mydir

Run "MSYS-20111123\msys\msys.bat"

> cd c:\mydir
> tar xf openssl-1.0.2a.tar.gz
That moment is very important - if you just unpack it by some another tool (I've used FAR manager first time) - it can handle wrong way symlinks and you will get error during compiling.
Actually on stackoverflow wrote that if you got bad symlinks - you can either fix some source files of tests or not build tests at all - but I think much simpler use unpacker from msys.
> cd ./openssl-1.0.2a

For x64:
    > perl Configure mingw64 no-shared no-asm --prefix=/C/OpenSSL-x64
For x32:
    > perl Configure mingw no-shared no-asm --prefix=/C/OpenSSL-x32
   
> make depend
> make
> make install


useful sources & links:

Documentation:
    https://github.com/openssl/openssl/blob/master/INSTALL.W32 (or just file 'INSTALL.W32' in openssl sources) for win32
    https://github.com/openssl/openssl/blob/master/INSTALL.W64 (or just file 'INSTALL.W64' in openssl sources) for win64
    https://wiki.openssl.org/index.php/Compilation_and_Installation (here are useful options table, by the way)
Here are a lot of useful advices, but part of them out-of-date, bcs of unsing unsupported masm:
    http://developer.covenanteyes.com/building-openssl-for-visual-studio/
Good explanation of mingw-building
    http://stackoverflow.com/questions/9379363/how-to-build-openssl-with-mingw-in-windows
Good advices about VisualStudio compiling and SAFESEH (somewhy he builds openssl with "no-idea" key (it means without IDEA algorithm))
    http://eran.geek.co.il/wp/archives/3897
Yet another typical guide, but here are described alternative way of dealing with symbols
    http://p-nand-q.com/programming/windows/building_openssl_with_visual_studio_2013.html
Here are something about zlib (I just don't need that):
    http://www.nu42.com/2014/04/building-openssl-101g-on-64-bit-windows.html
    http://stackoverflow.com/questions/23772816/when-do-i-need-zlib-in-openssl
Here are about runtime library:
    http://stackoverflow.com/questions/18486243/how-do-i-build-openssl-statically-linked-against-windows-runtime
Here are a bit more about mingw building:
    https://wiki.qt.io/Compiling-OpenSSL-with-MinGW
Here are (in the end) interesting idea - put openssl into visual studio internals, to not care about paths to includes & libraries every time (I solve it by property sheets, but this method is interesting)
    http://www.libtorrent.org/building.html
Here are somebody wrote scripts for building.
    http://www.michaelboman.org/how-to/building-openssl-on-windows
    http://www.lenholgate.com/blog/2012/08/building-openssl-for-x86-and-x64-on-windows-for-side-by-side-deployment.html
Here are yet another compiling problem - it seems somebody didn't turn off asm on x86
    http://stackoverflow.com/questions/15304327/building-openssl-vc-static-64bit-libs
ClamAV out-of-date building article (bcs of unsing unsupported masm). Nothing interesting. Just leave here for myself.
    http://blog.clamav.net/2014/07/compiling-openssl-for-windows.html
Yet another typical guide, but with free VisualStudio
    http://botsikas.blogspot.ru/2012/10/building-openssl-on-windows.html
Yet another typical guide, with some extra errors described
    http://wiki.openuru.org/index.php?title=Build_OpenSSL_with_MS_Visual_C%2B%2B
Yet another guide. From 2007. Maybe here will be something interesting.
    http://www.ski-epic.com/2007_notes_on_openssl/index.html
Yet another guide. Described running tests.
    http://www.poweradmin.com/blog/building-openssl-with-visual-studio-2012-for-32-or-64-bit/


possible errors (for googling people):

If you unpack archive for mingw with wrong handling of symlinks - you will get next message during compiling:
--------------------------------------------------------------------------------
make[1]: Entering directory `/c/mydir/test'
gcc -I.. -I../include  -DOPENSSL_THREADS -D_MT -DDSO_WIN32 -DL_ENDIAN -O3 -Wall
-DWIN32_LEAN_AND_MEAN -DUNICODE -D_UNICODE   -c -o md2test.o md2test.c
md2test.c:1:10: error: expected '=', ',', ';', 'asm' or '__attribute__' before '
.' token
 dummytest.c
          ^
make[1]: *** [md2test.o] Error 1
make[1]: Leaving directory `/c/mydir/test'
make: *** [build_tests] Error 1
--------------------------------------------------------------------------------
what to do: use tar from msys to unpacking archive


If you are trying to compile with visual studio but not from it's command prompt:
--------------------------------------------------------------------------------
'nmake' is not recognized as an internal or external command
--------------------------------------------------------------------------------
what to do: use visual studio command prompt


If you are trying to compile with mingw from cmd (or far manager)
--------------------------------------------------------------------------------
RC4_CHUNK is unsigned long long
e_os2.h => include/openssl/e_os2.h
"making $target in $dir..."
'TOP' is not recognized as an internal or external command,
operable program or batch file.
make: *** [links] Error 1
--------------------------------------------------------------------------------
what to do: use msys to compiling with mingw


If you are trying to compile with visual studio with masm
--------------------------------------------------------------------------------
Assembling: tmp32\sha1-586.asm
tmp32\sha1-586.asm(1432) : error A2070:invalid instruction operands
tmp32\sha1-586.asm(1576) : error A2070:invalid instruction operands
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\ml.EXE"' : return code '0x1'   
Stop.
--------------------------------------------------------------------------------
what to do: either use nasm, either no-asm

5 comments:

  1. I tried the steps to build openss dynamic release for Win 64-bit. I am using VS 2013. When I type: "nmake -f ms\ntdll.mak" I get this error:
    "x86_64-mont.obj : error LNK2005: bn_mul_mont already defined in bn_asm.obj
    tmp32dll\rsaz_exp.obj : fatal error LNK1112: module machine type 'X86' conflicts
    with target machine type 'x64'
    NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 12.0
    \VC\BIN\amd64\link.EXE"' : return code '0x458'
    Stop."

    Here is a screen shot:
    http://i61.tinypic.com/2gv0mpx.png

    Can you help plz.

    ReplyDelete
    Replies
    1. Hi. Sorry for late answer - I'm not often here.

      "tmp32dll\rsaz_exp.obj : fatal error LNK1112: module machine type 'X86' conflicts with target machine type 'x64'"

      You are linking x86 & x64 code. Usually it happens bcs previous compilation was, for example, for x86, and current for x64 - and some obj files didn't recompiled.
      So, you can make either
      1) 'nmake clean' & try to compile one more time
      or
      2) take source code archive, unpack it to new directory and compile here x64 version)

      Delete
  2. This post was incredibly helpful, thank you very much.

    ReplyDelete