OSSL-GUIDE-TLS-INTRODUCTION(7ossl) OpenSSL OSSL-GUIDE-TLS-INTRODUCTION(7ossl)
NAME
ossl-guide-tls-introduction - OpenSSL Guide: An introduction to SSL/TLS
in OpenSSL
INTRODUCTION
This page will provide an introduction to some basic SSL/TLS concepts
and background and how it is used within OpenSSL. It assumes that you
have a basic understanding of TCP/IP and sockets.
WHAT IS TLS?
TLS stands for Transport Layer Security. TLS allows applications to
securely communicate with each other across a network such that the
confidentiality of the information exchanged is protected (i.e. it
prevents eavesdroppers from listening in to the communication).
Additionally it protects the integrity of the information exchanged to
prevent an attacker from changing it. Finally it provides
authentication so that one or both parties can be sure that they are
talking to who they think they are talking to and not some imposter.
Sometimes TLS is referred to by its predecessor's name SSL (Secure
Sockets Layer). OpenSSL dates from a time when the SSL name was still
in common use and hence many of the functions and names used by OpenSSL
contain the "SSL" abbreviation. Nonetheless OpenSSL contains a fully
fledged TLS implementation.
TLS is based on a client/server model. The application that initiates a
communication is known as the client. The application that responds to
a remotely initiated communication is the server. The term "endpoint"
refers to either of the client or the server in a communication. The
term "peer" refers to the endpoint at the other side of the
communication that we are currently referring to. So if we are
currently talking about the client then the peer would be the server.
TLS is a standardised protocol and there are numerous different
implementations of it. Due to the standards an OpenSSL client or server
is able to communicate seamlessly with an application using some
different implementation of TLS. TLS (and its predecessor SSL) have
been around for a significant period of time and the protocol has
undergone various changes over the years. Consequently there are
different versions of the protocol available. TLS includes the ability
to perform version negotiation so that the highest protocol version
that the client and server share in common is used.
TLS acts as a security layer over some lower level transport protocol.
Typically the transport layer will be TCP.
SSL AND TLS VERSIONS
SSL was initially developed by Netscape Communications and its first
publicly released version was SSLv2 in 1995. Note that SSLv1 was never
publicly released. SSLv3 came along quickly afterwards in 1996.
Subsequently development of the protocol moved to the IETF which
released the first version of TLS (TLSv1.0) in 1999 as RFC2246. TLSv1.1
was released in 2006 as RFC4346 and TLSv1.2 came along in 2008 as
RFC5246. The most recent version of the standard is TLSv1.3 which was
released in 2018 as RFC8446.
Today TLSv1.3 and TLSv1.2 are the most commonly deployed versions of
the protocol. The IETF have formally deprecated TLSv1.1 and TLSv1.0, so
anything below TLSv1.2 should be avoided since the older protocol
versions are susceptible to security problems.
OpenSSL does not support SSLv2 (it was removed in OpenSSL 1.1.0).
Support for SSLv3 is available as a compile time option - but it is not
built by default. Support for TLSv1.0, TLSv1.1, TLSv1.2 and TLSv1.3
are all available by default in a standard build of OpenSSL. However
special run-time configuration is required in order to make TLSv1.0 and
TLSv1.1 work successfully.
OpenSSL will always try to negotiate the highest protocol version that
it has been configured to support. In most cases this will mean either
TLSv1.3 or TLSv1.2 is chosen.
CERTIFICATES
In order for a client to establish a connection to a server it must
authenticate the identify of that server, i.e. it needs to confirm that
the server is really the server that it claims to be and not some
imposter. In order to do this the server will send to the client a
digital certificate (also commonly referred to as an X.509
certificate). The certificate contains various information about the
server including its full DNS hostname. Also within the certificate is
the server's public key. The server operator will have a private key
which is linked to the public key and must not be published.
Along with the certificate the server will also send to the client
proof that it knows the private key associated with the public key in
the certificate. It does this by digitally signing a message to the
client using that private key. The client can verify the signature
using the public key from the certificate. If the signature verifies
successfully then the client knows that the server is in possession of
the correct private key.
The certificate that the server sends will also be signed by a
Certificate Authority. The Certificate Authority (commonly known as a
CA) is a third party organisation that is responsible for verifying the
information in the server's certificate (including its DNS hostname).
The CA should only sign the certificate if it has been able to confirm
that the server operator does indeed have control of the server
associated with its DNS hostname and that the server operator has
control of the private key.
In this way, if the client trusts the CA that has signed the server's
certificate and it can verify that the server has the right private key
then it can trust that the server truly does represent the DNS hostname
given in the certificate. The client must also verify that the hostname
given in the certificate matches the hostname that it originally sent
the request to.
Once all of these checks have been done the client has successfully
verified the identify of the server. OpenSSL can perform all of these
checks automatically but it must be provided with certain information
in order to do so, i.e. the set of CAs that the client trusts as well
as the DNS hostname for the server that this client is trying to
connect to.
Note that it is common for certificates to be built up into a chain.
For example a server's certificate may be signed by a key owned by a an
intermediate CA. That intermediate CA also has a certificate
containing its public key which is in turn signed by a key owned by a
root CA. The client may only trust the root CA, but if the server sends
both its own certificate and the certificate for the intermediate CA
then the client can still successfully verify the identity of the
server. There is a chain of trust between the root CA and the server.
By default it is only the client that authenticates the server using
this method. However it is also possible to set things up such that the
server additionally authenticates the client. This is known as "client
authentication". In this approach the client will still authenticate
the server in the same way, but the server will request a certificate
from the client. The client sends the server its certificate and the
server authenticates it in the same way that the client does.
TRUSTED CERTIFICATE STORE
The system described above only works if a chain of trust can be built
between the set of CAs that the endpoint trusts and the certificate
that the peer is using. The endpoint must therefore have a set of
certificates for CAs that it trusts before any communication can take
place. OpenSSL itself does not provide such a set of certificates.
Therefore you will need to make sure you have them before you start if
you are going to be verifying certificates (i.e. always if the endpoint
is a client, and only if client authentication is in use for a server).
Fortunately other organisations do maintain such a set of certificates.
If you have obtained your copy of OpenSSL from an Operating System (OS)
vendor (e.g. a Linux distribution) then normally the set of CA
certificates will also be distributed with that copy.
You can check this by running the OpenSSL command line application like
this:
openssl version -d
This will display a value for OPENSSLDIR. Look in the certs sub
directory of OPENSSLDIR and check its contents. For example if
OPENSSLDIR is "/usr/local/ssl", then check the contents of the
"/usr/local/ssl/certs" directory.
You are expecting to see a list of files, typically with the suffix
".pem" or ".0". If they exist then you already have a suitable trusted
certificate store.
If you are running your version of OpenSSL on Windows then OpenSSL
(from version 3.2 onwards) will use the default Windows set of trusted
CAs.
If you have built your version of OpenSSL from source, or obtained it
from some other location and it does not have a set of trusted CA
certificates then you will have to obtain them yourself. One such
source is the Curl project. See the page
<https://curl.se/docs/caextract.html> where you can download trusted
certificates in a single file. Rename the file to "cert.pem" and store
it directly in OPENSSLDIR. For example if OPENSSLDIR is
"/usr/local/ssl", then save it as "/usr/local/ssl/cert.pem".
You can also use environment variables to override the default location
that OpenSSL will look for its trusted certificate store. Set the
SSL_CERT_PATH environment variable to give the directory where OpenSSL
should looks for its certificates or the SSL_CERT_FILE environment
variable to give the name of a single file containing all of the
certificates. See openssl-env(7) for further details about OpenSSL
environment variables. For example you could use this capability to
have multiple versions of OpenSSL all installed on the same system
using different values for OPENSSLDIR but all using the same trusted
certificate store.
You can test that your trusted certificate store is setup correctly by
using it via the OpenSSL command line. Use the following command to
connect to a TLS server:
openssl s_client www.openssl.org:443
Once the command has connected type the letter "Q" followed by
"<enter>" to exit the session. This will print a lot of information on
the screen about the connection. Look for a block of text like this:
SSL handshake has read 4584 bytes and written 403 bytes
Verification: OK
Hopefully if everything has worked then the "Verification" line will
say "OK". If its not working as expected then you might see output
like this instead:
SSL handshake has read 4584 bytes and written 403 bytes
Verification error: unable to get local issuer certificate
The "unable to get local issuer certificate" error means that OpenSSL
has been unable to find a trusted CA for the chain of certificates
provided by the server in its trusted certificate store. Check your
trusted certificate store configuration again.
Note that s_client is a testing tool and will still allow you to
connect to the TLS server regardless of the verification error. Most
applications should not do this and should abort the connection in the
event of a verification error.
IMPORTANT OBJECTS FOR AN OPENSSL TLS APPLICATION
A TLS connection is represented by the SSL object in an OpenSSL based
application. Once a connection with a remote peer has been established
an endpoint can "write" data to the SSL object to send data to the
peer, or "read" data from it to receive data from the server.
A new SSL object is created from an SSL_CTX object. Think of an SSL_CTX
as a "factory" for creating SSL objects. You can create a single
SSL_CTX object and then create multiple connections (i.e. SSL objects)
from it. Typically you can set up common configuration options on the
SSL_CTX so that all the SSL object created from it inherit the same
configuration options.
Note that internally to OpenSSL various items that are shared between
multiple SSL objects are cached in the SSL_CTX for performance reasons.
Therefore it is considered best practice to create one SSL_CTX for use
by multiple SSL objects instead of having one SSL_CTX for each SSL
object that you create.
Each SSL object is also associated with two BIO objects. A BIO object
is used for sending or receiving data from the underlying transport
layer. For example you might create a BIO to represent a TCP socket.
The SSL object uses one BIO for reading data and one BIO for writing
data. In most cases you would use the same BIO for each direction but
there could be some circumstances where you want them to be different.
It is up to the application programmer to create the BIO objects that
are needed and supply them to the SSL object. See
ossl-guide-tls-client-block(7) for further information.
Finally, an endpoint can establish a "session" with its peer. The
session holds various TLS parameters about the connection between the
client and the server. The session details can then be reused in a
subsequent connection attempt to speed up the process of connecting.
This is known as "resumption". Sessions are represented in OpenSSL by
the SSL_SESSION object. In TLSv1.2 there is always exactly one session
per connection. In TLSv1.3 there can be any number per connection
including none.
PHASES OF A TLS CONNECTION
A TLS connection starts with an initial "set up" phase. The endpoint
creates the SSL_CTX (if one has not already been created) and
configures it.
A client then creates an SSL object to represent the new TLS
connection. Any connection specific configuration parameters are then
applied and the underlying socket is created and associated with the
SSL via BIO objects.
A server will create a socket for listening for incoming connection
attempts from clients. Once a connection attempt is made the server
will create an SSL object in the same way as for a client and associate
it with a BIO for the newly created incoming socket.
After set up is complete the TLS "handshake" phase begins. A TLS
handshake consists of the client and server exchanging a series of TLS
handshake messages to establish the connection. The client starts by
sending a "ClientHello" handshake message and the server responds with
a "ServerHello". The handshake is complete once an endpoint has sent
its last message (known as the "Finished" message) and received a
Finished message from its peer. Note that this might occur at slightly
different times for each peer. For example in TLSv1.3 the server always
sends its Finished message before the client. The client later responds
with its Finished message. At this point the client has completed the
handshake because it has both sent and received a Finished message. The
server has sent its Finished message but the Finished message from the
client may still be in-flight, so the server is still in the handshake
phase. It is even possible that the server will fail to complete the
handshake (if it considers there is some problem with the messages sent
from the client), even though the client may have already progressed to
sending application data. In TLSv1.2 this can happen the other way
around, i.e. the server finishes first and the client finishes second.
Once the handshake is complete the application data transfer phase
begins. Strictly speaking there are some situations where the client
can start sending application data even earlier (using the TLSv1.3
"early data" capability) - but we're going to skip over that for this
basic introduction.
During application data transfer the client and server can read and
write data to the connection freely. The details of this are typically
left to some higher level application protocol (for example HTTP). Not
all information exchanged during this phase is application data. Some
protocol level messages may still be exchanged - so it is not
necessarily the case that, just because the underlying socket is
"readable", that application data will be available to read.
When the connection is no longer required then it should be shutdown. A
shutdown may be initiated by either the client or the server via a
message known as a "close_notify" alert. The client or server that
receives a close_notify may respond with one and then the connection is
fully closed and application data can no longer be sent or received.
Once shutdown is complete a TLS application must clean up by freeing
the SSL object.
FURTHER READING
See ossl-guide-tls-client-block(7) to see an example of applying these
concepts in order to write a simple TLS client based on a blocking
socket. See ossl-guide-quic-introduction(7) for an introduction to
QUIC in OpenSSL.
SEE ALSO
ossl-guide-introduction(7), ossl-guide-libraries-introduction(7),
ossl-guide-libssl-introduction(7), ossl-guide-tls-client-block(7),
ossl-guide-quic-introduction(7)
COPYRIGHT
Copyright 2023 The OpenSSL Project Authors. All Rights Reserved.
Licensed under the Apache License 2.0 (the "License"). You may not use
this file except in compliance with the License. You can obtain a copy
in the file LICENSE in the source distribution or at
<https://www.openssl.org/source/license.html>.
3.3.2 2024-09-04
OSSL-GUIDE-TLS-INTRODUCTION(7ossl)
openssl 3.3.2 - Generated Tue Oct 1 16:40:17 CDT 2024