wget and curl for MSYS

Post date: Sep 16, 2012 5:2:50 AM

Build information was modified from original at Fragrant Memories.

wget and curl are utilities for downloading files and webpages via the command line.

You can install a pre-built wget 1.12 from msys:

mingw-get install msys-wget

The instructions below will install a newer version of wget with some bugfixes.

Prerequisites

zlib

OpenSSL

libssh2 (It is optional, remove --with-libssl2=/usr/local option to do without it).

You can use 7-zip for extracting files instead of tar.

Download wget source

Download the latest release of wget from the GNU FTP site, currently wget-1.18.tar.gz.

Move the download to your home directory.

Extract using tar:

cd

tar xvfz wget-1.18.tar.gz

Change to the build directory:

cd wget-1.18/

Build wget

Configure the makefile to use pthreads, OpenSSL, and disable native language support (NLS). This takes a few minutes:

./configure --enable-threads=win32 --disable-nls --with-ssl=openssl CC=-std=c99

The default --prefix option was /usr/local, change this if desired to install elsewhere.

The last version of MinGW changed some headers causing problems. TODO: fix this error:

spawn-pipe.c: In function 'create_pipe':

spawn-pipe.c:193:41: error: 'environ' undeclared (first use in this function)

(const char **) environ);

Run make, then install wget:

make

make install

Use certificate bundle

OpenSSL based programs can check a certificate bundle file when making secure connections.

If you built openssl yourself:

cd /usr/local/ssl/certs/

If you used the msys-openssl package:

cd /var/ssl/certs/

Download the file from the curl website using wget:

wget --no-check-certificate http://curl.haxx.se/ca/cacert.pem

If you don't have wget, you can copy cacert.pem from your browser: http://curl.haxx.se/ca/cacert.pem

Verify the fingerprint:

openssl x509 -noout -fingerprint -in /var/ssl/certs/cacert.pem

Add this line to your .profile:

export SSL_CERT_FILE=/usr/local/ssl/certs/cacert.pem

Download curl source

You could use wget to download curl source, or do it the "old-fashioned way" from web browser at curl.haxx.se.

cd

wget https://curl.haxx.se/download/curl-7.51.0.tar.gz

tar xvfz curl-7.51.0.tar.gz

Build curl

I needed to set an environment variable for SSL support to be included, it shouldn't have been needed:

export LIBS=-lz

Change to the directory, configure for MinGW, make, and install:

cd curl-7.51.0

./configure --build=i686-pc-mingw32 --with-zlib --with-libssh2 --with-ssl=/usr/local

make

make install

You can run configure --help to see all configuration options. For example, --disable-shared will build curl with static libraries. This increases the file size, but removes dependencies on DLLs.

TODO: more options, like IPv6, Kerberos, international domain names

What was installed?

• • /usr/local/lib/

  • libcurl.la

  • libcurl.a

  • pkgconfig/*.pc

  • /usr/local/include

  • curl/

  • /usr/local/bin/

  • curl.exe

  • curl-config

Using curl

Here's an example to download a file from a redirected URL:

curl -LOJ https://api.github.com/repos/SFML/SFML/tarball/master

-L, --location

If the server returns a 3XX redirect code, redo the request at the new location

-O, --remote-name

Write output to a local file

-J, --remote-header-name

Use the server-specified filename instead of extracting a filename from the URL.