Skip to content

Generate keys

ssh-keygen -C "$(whoami)@$(hostname)-$(date -I)" -t ed25519
ssh-keygen -t rsa -b 4096
  • -C Comment to identify the key. Add user name and date for easier management.
  • -b The number of bits.
  • -t Type of key.
Note: Types of keys Types of keys: - `ed25519`: Newer and *should* be more secure. **Recomended** - `rsa`: Works with old software. *Should* be secure. - `ecdsa`: *Could* be compromised as uses NIST P-256 curve. [Source](https://safecurves.cr.yp.to/index.html). - `dsa`: **insecure**

At this point in time no one can tell which is more "secure" because they are pretty much "unbreakable".

Add a password to the key whenever possible.

This command will generate two files a private ~/.ssh/id_ed25519 and a public ~/.ssh/id_ed25519.pub; the public can be shared the private must be kept a secret.

Copy the keys to the server

Using an automated method (you need to have access to the server through SSH):

ssh-copy-id -i /path/to/key user@server

To manually copy the key to the server you have to put the content of the ~/.ssh/id_ed25519.pub file into the server's ~/.ssh/authorized-keys.

File permissions

chmod 700 ~/.ssh
chmod 400 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/known_hosts

Note: authorized_keys may need to be read from other users or programs.

Configure the server

sshd service

Running ssh as a socket (start an ssh instance when there is a connection) is generally the recommended way unless you have many users as this method spans a ssh instance per connection systemctl --full | grep ssh. It also allows you to kill individual connections. Source

Disable and stop ssh

sudo systemctl disable sshd
sudo systemctl stop sshd

Edit the IP address in which the ssh socket listens with /etc/systemd/system/sshd.socket.d/override.conf or sudo EDITOR=vim systemctl edit sshd.socket.

[Socket]
ListenStream=192.168.1.1:22
FreeBind=true

Enable ssh socket.

sudo systemctl start sshd.socket
sudo systemctl enable sshd.socket

Changing the port of SSH

By default SSH runs in port 22, a privileged port; this means only root can run a program at that port. If you change the port to port 4567 any user can run anything in that port implying that someone with access can crash SSH and run a malicious service on that port.

If you want to still change the port add the following to iptables. Source

sudo iptables -t nat -A PREROUTING -p tcp --dport 4567 -j CONNMARK --set-mark 4567
sudo iptables -t nat -A PREROUTING -p tcp --dport 4567 -j REDIRECT --to-port 22

sudo iptables -A INPUT -p tcp --dport 22 -m connmark ! --mark 4567 -j REJECT
sudo iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT

/etc/ssh/sshd_config

sshd_config Some options are commented out, that may mean they are the defaults. 
## Service listening options. Useless when using a socket.
#Port 22
#AddressFamily any ## inet (IPv4); inet6 (IPv6) or any for both
#ListenAddress 0.0.0.0 ## By default SSH listens on all interfaces
#ListenAddress :: ## Same but for IPv6

## Default Private SSH keys for the server
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_dsa_key ## Insecure
#HostKey /etc/ssh/ssh_host_ecdsa_key ## Insecure

# Ciphers and keying
## maximum amount of data that may be transmitted before the session key is
## renegotiated
#RekeyLimit default none

# Logging
## Facility code used when logging messages
#SyslogFacility AUTH ## Arch Linux
SyslogFacility AUTHPRIV ## CentOS

## LogLevel VERBOSE logs user's key fingerprint on login. Needed to have a
## clear audit track of which key was using to log in.
## Logs at ????
LogLevel VERBOSE

# Authentication:

## The server disconnects after this time if the user has not successfully
## logged in.
#LoginGraceTime 2m

## Allow login as root. without-password (public keys); forced-commands-only
## (only certain commands); yes; no (preferred)
PermitRootLogin no

## Check file modes and ownership of the user's files and home directory
## before accepting login.
#StrictModes yes

## Maximum number of authentication attempts permitted per connection.
MaxAuthTries 1

## Maximum number of open sessions (*shells*) per IP.
#MaxSessions 10

## Allow public keys to authenticate.
#PubkeyAuthentication yes

# The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
# but this is overridden so installations will only check .ssh/authorized_keys
AuthorizedKeysFile  .ssh/authorized_keys

## Specifies a file that lists principal names that are accepted for certificate
## authentication. Use with CA issued certificates.
#AuthorizedPrincipalsFile none

## Specifies a program to be used to look up the user's public keys.
#AuthorizedKeysCommand none

## Specifies the user under whose account the AuthorizedKeysCommand is run.
#AuthorizedKeysCommandUser nobody

## Specifies whether rhosts or /etc/hosts.equiv authentication together with
## successful public key client host authentication is allowed. This is old
## and is replaced with public keys.
# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no

# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no

## Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

## Disallow passwords for authentication.
PasswordAuthentication no

## Permit log in in accounts with no password.
#PermitEmptyPasswords no

## Disallow keyboard interactive passwords. Set to yes for TOTP.
ChallengeResponseAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no
#KerberosUseKuserok yes

# GSSAPI options
## Used for kerberos authentication.
## Can be disabled if not used.
GSSAPIAuthentication no
#GSSAPICleanupCredentials no
GSSAPIStrictAcceptorCheck no
#GSSAPIKeyExchange no
#GSSAPIEnablek5users no

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
# WARNING: 'UsePAM no' is not supported in Red Hat Enterprise Linux
# and may cause several problems.
## Needed for TOTP
UsePAM yes

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
#PrintMotd yes
#PrintLastLog yes
#UseLogin no
#UsePrivilegeSeparation sandbox
#PermitUserEnvironment no
#Compression delayed

## Used to monitor whether the client is still available.
#TCPKeepAlive yes

## Client time out. 0 to disable.
ClientAliveInterval 60

## Number of keep alive packages to send.
ClientAliveCountMax 0

## The SSH server will look up the remote hostname and check with DNS if the resolved IP for the remote hostname matches.
## Disabling can speed up connections.
UseDNS no

#ShowPatchLevel no
#PidFile /var/run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

## Allow only these users.
AllowUsers yu user admin john

## Allow only the users in these groups.
AllowGroups sshusers

## All users except these.
DenyUsers root

## Only use SSH protocol 2.
Protocol 2

## Banner (motd) to display when login.
#Banner none
Banner /etc/issue.net


## Only accept specific KEX (Key Exchange) algorithms.
# ssh -Q kex
KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group16-sha512,curve25519-sha256,diffie-hellman-group-exchange-sha256,sntrup761x25519-sha512@openssh.com,curve25519-sha256,diffie-hellman-group-exchange-sha256,sntrup761x25519-sha512@openssh.com

## Specifies the ciphers allowed.
# ssh -Q cipher
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr

## Specifies the available MAC (message authentication code) algorithms.
# ssh -Q mac
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com

## Specifies the host key signature algorithms that the server offers.
# ssh -Q HostKeyAlgorithms
HostKeyAlgorithms ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256



## Override default of no subsystems [leave].
#Subsystem  sftp    /usr/lib/ssh/sftp-server ## Arch Linux
#Subsystem  sftp    /usr/lib/openssh/sftp-server ## Debian
#Subsystem  sftp    /usr/libexec/openssh/sftp-server ## CentOS

## Log sftp level file access (read/write/etc.) that would not be easily
## logged otherwise.
## Logs at ????
Subsystem sftp  /usr/lib/ssh/sftp-server -f AUTHPRIV -l INFO

## Multiple authentication methods (TOTP) (publickey and TOTP code).

## Publickey and TOTP code. Depending on `/etc/pam.d/sshd` it may ask for
## the user password too.
AuthenticationMethods publickey,keyboard-interactive

## Publickey and user password. Not recommended.
#AuthenticationMethods publickey,password

## Don't know if it works. Not recommended.
#AuthenticationMethods "publickey,password" "publickey,keyboard-interactive"
#PasswordAuthentication yes

## Accept locale-related environment variables.
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS

## Other [leave].
#RhostsRSAAuthentication no
#RSAAuthentication yes
#KeyRegenerationInterval 3600
#ServerKeyBits 1024


# Example of overriding settings on a per-user basis
## More settings????
#Match User anoncvs
#  X11Forwarding no
#  AllowTcpForwarding no
#  PermitTTY no
#  ForceCommand cvs server

## rsync user
Match User rsync
  ChrootDirectory /home/%u ## Must be owned by root??
  PermitTTY no
  ForceCommand rsync
  AllowTcpForwarding no
  X11Forwarding no

Restart ssh if you are using a service instead of a socket.

systemctl restart sshd

Regenerate host SSH keys

Remove old keys.

rm -v /etc/ssh/ssh_host_*

Generate rsa and ed25519 host keys.

ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ""

Generate all missing keys with ssh-keygen -A

Obtaining the public key fingerprint for the server

ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

~/.ssh/authorized-keys

Some examples from man 8 sshd where you can find all options.

# Plain key, no restrictions
ssh-rsa ...

# Forced command, disable PTY and all forwarding
restrict,command="dump /home" ssh-rsa ...

# Restriction of ssh -L forwarding destinations
permitopen="192.0.2.1:80",permitopen="192.0.2.2:25" ssh-rsa ...

# Restriction of ssh -R forwarding listeners
permitlisten="localhost:8080",permitlisten="[::1]:22000" ssh-rsa ...

# Configuration for tunnel forwarding
tunnel="0",command="sh /etc/netstart tun0" ssh-rsa ...

Configure the client

For global options edit /etc/ssh/ssh_config

Host *
    IdentitiesOnly yes
    MACs umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com
    Ciphers chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
    HostKeyAlgorithms ssh-ed25519-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,sk-ssh-ed25519@openssh.com,rsa-sha2-512,rsa-sha2-256
    KexAlgorithms sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256

For user specific options edit ~/.ssh/config

AddKeysToAgent  yes

## All hosts
Host *
    IdentitiesOnly yes

Host server
    HostName 192.168.1.45
    User yu
    Port 4567
    IdentityFile /home/yu/keys/server

All options explained in ssh_config(5).

Map a remote port

man 1 ssh

Socket forwarding

ssh -L /var/mysql/mysql.sock:/var/mysql/mysql.sock username@hostname

Port forwarding

Access a remote service from your local system.

-L [bind_address:]port:host:hostport
-L [bind_address:]port:remote_socket
-L local_socket:host:hostport
-L local_socket:remote_socket

ssh -L local-port:remote-hostname:remote-port username@hostname

Example to forward port 110 in the server to port 1100 in the client.

ssh -L 1100:127.0.0.1:110 mail.example.local
ssh -L 1100:mail.example.local:110 mail.example.local

Remote forwarding

Access a local service from a remote system.

-R [bind_address:]port:host:hostport
-R [bind_address:]port:local_socket
-R remote_socket:host:hostport
-R remote_socket:local_socket
-R [bind_address:]port

ssh -R local-port:hostname:client-port username@hostname

Example: the client is listening for connections on localhost:8080 and we want to connect from the server server.example.local at localhost:9001:

ssh -R 9001:localhost:8080 server.example.local

You can map the forwarded port to the server address to be accessible by your local network with:

ssh -R server.example.local:9001:localhost:8080 server.example.local

It has to be allowed in /etc/ssh/sshd_config with:

GatewayPorts clientspecified

Dynamic forwarding (SOCKS proxy)

ssh -D [bind_address:]port

ssh -D 8000 server.example.com

Background forwarding

-fNn will create (map) a tunnel in the background.

ssh -fNn -L 1100:127.0.0.1:110 mail.example.local
ssh -fNn -L localhost:1100:localhost:110 mail.example.local

Finish the connection with:

kill -QUIT "$( pgrep -xf "ssh -fNn -L 1100:127.0.0.1:110 mail.example.local" )"

SSH agent

Start ssh-agent

Manually.

eval `ssh-agent -s`

Or

eval "$(ssh-agent -s)"

In .bashrc or .zshrc.

[[ $SSH_AGENT_PID ]] || eval `ssh-agent -s` > /dev/null && ssh-add /path/to/key

In .xinitrc.

exec ssh-agent sway
#exec ssh-agent i3
#exec ssh-agent openbox ## Openbox
#exec ssh-agent openbox-session ## Openbox alternative
#exec ssh-agent startkde ## KDE

Add keys

Add one key

ssh-add /path/to/key
ssh-add /path/to/key -t 3600 ## For 1 hour

man 1 ssh-add

Remove keys

Remove one key.

ssh-add -d /path/to/key

Remove all keys.

ssh-add -D

X11Forwarding

WIP

ssh -Y server.example.local firefox

Port knocking

WIP

$IPT -N stage1
$IPT -A stage1 -m recent --remove --name knock
$IPT -A stage1 -p tcp --dport 3456 -m recent --set --name knock2

$IPT -N stage2
$IPT -A stage2 -m recent --remove --name knock2
$IPT -A stage2 -p tcp --dport 2345 -m recent --set --name heaven

$IPT -N door
$IPT -A door -m recent --rcheck --seconds 5 --name knock2 -j stage2
$IPT -A door -m recent --rcheck --seconds 5 --name knock -j stage1
$IPT -A door -p tcp --dport 1234 -m recent --set --name knock

$IPT -A INPUT -m --state ESTABLISHED,RELATED -j ACCEPT
$IPT -A INPUT -p tcp --dport 22 -m recent --rcheck --seconds 5 --name heaven -j ACCEPT
$IPT -A INPUT -p tcp --syn -j door

Two factor authentication (TOTP)

What we need: - pam-oath. PAM module to authenticate with TOTP codes. - A seed used to generate TOTP codes by the client and server. - A users file to define TOTP type, username, pin (optional) and seed.

Install pam-oath

More information in pam-oath.

Help: Installation [Archlinux](https://www.nongnu.org/oath-toolkit/pam_oath.html) 
pacman -S oath-toolkit
Debian 
apt install libpam-oath
CentOS 
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
rpm -Uvh epel-release-latest-7.noarch.rpm
yum install pam_oath

Create a users file

Generate a seed. This seed is in hex format.

head -15 /dev/urandom | sha512sum | cut -b 1-30
---
6b400f7b322b33a98b9ace4dbeff3e

Create the users file /etc/users.oath.

HOTP/T30 user - 6b400f7b322b33a98b9ace4dbeff3e
Users file documentation - `HOTP/T30` is the token type (see below). - `user` is the user for which the token is for. - `-` is the PIN, which goes before the TOTP code. If the PIN is `4567` and the totp code is `72365962` the code ssh will ask for is `456772365962` - `6b400f7b322b33a98b9ace4dbeff3e` is the seed in hex.

Sources

- -

Overview

The users file contains the secret information about users, PINs, and token keys on the server. It's location is configured via the `OTPAuthUsersFile` directive. This file is updated (rewritten) whenever the server needs to save new information. Therefore, you should only edit it when the server is stopped (or no authentication is occurring). You will need to create this file manually and be sure to set the permissions correctly (see below). Each line defines one user+token combination. Currently a user may only have a single associated token. Users may have an arbitrary length PIN, or a dash ("-") for no PIN, or a plus ("+") if the PIN is verified externally using the configured `OTPAuthPINAuthProvider` list (see [Configuration](Configuration) for details on external PIN verification). It is strongly recommended to set all PIN's to a plus sign and configure a `OTPAuthPINAuthProvider` for PIN verification. When a user has a PIN, it must precede the one-time password in the HTTP password field. For example, if a user has PIN "1234" and the token generates the one-time password of "567890", then the user must enter "1234567890" as their password for HTTP authentication. Tokens must be configured as event tokens or time tokens and of course the token's key must be entered. In addition, you may optionally specify the initial counter value (for event tokens) or relative time period offset adjustment (for time tokens); both values default to zero if not specified. For time based tokens, the length of a single time interval is specified as part of the token type (see below). See [OneTimePasswords](OneTimePasswords) and [Tokens](Tokens) for more information about how one-time passowords and tokens work.

Permissions

The Apache web server must have permission to create files in the same directory as this file, and to delete (overwrite) this file with an updated copy. This is because the way **mod\_authn\_otp** updates the file is by writing out a new version under a temporary name and then renaming the temporary file to the real one. For example, on [openSuSE Linux](http://www.opensuse.org/) this means the file and parent directory should be owned by user `wwwrun` and group `www`. If it's not already obvious, because it contains sensitive security information **never put this file where it could be publicly visible**. For example, it should probably be in a directory that is only readable by the Apache process.

Format

An example users file is included in the distribution and is reproduced here: 
#
# Example users file for mod_authn_otp
#
# Blank lines and lines starting with '#' are ignored. Fields are whitespace-separated.
#
# Fields:
#
#   1. Token Type         See below
#   2. Username           User's username
#   3. PIN                User's PIN, or "-" if user has no PIN, or "+" to verify PIN via "OTPAuthPINAuthProvider"
#   4. Token Key          Secret key for the token algorithm (see RFC 4226)
#   5. Counter/Offset     Next expected counter value (event tokens) or counter offset (time tokens)
#   6. Failure counter    Number of consecutive wrong OTP's provided by this users (for "OTPAuthMaxOTPFailure")
#   7. Last OTP           The previous successfully used one-time password
#   8. Time of Last OTP   Local timestamp when the last OTP was generated (in the form 2009-06-12T17:52:32L)
#   9. Last IP address    IP address used during the most recent successful attempt
#
#   Fields 5 and beyond are optional. Fields 6 and beyond should be omitted for new users.
#
# Token Type Field:
#
#   This field contains a string in the format: ALGORITHM [ / COUNTERINFO [ / DIGITS ] ]
#
#   The ALGORITHM is either "HOTP" (RFC 4226) or "MOTP" (http://motp.sourceforge.net/).
#
#   The COUNTERINFO is either "E" for an event-based token, or "TNN" for a time based token
#   where "NN" is the number of seconds in one time interval. For HOTP, the default is "E";
#   for MOTP, the default is "T10".
#
#   The DIGITS is the number of digits in the one-time password; the default is six.
#
#   Examples:
#
#       HOTP            - HOTP event-based token with six digit OTP
#       HOTP/E          - HOTP event-based token with six digit OTP
#       HOTP/E/8        - HOTP event-based token with eight digit OTP
#       HOTP/T30        - HOTP time-based token with 30 second interval and six digit OTP
#       HOTP/T60        - HOTP time-based token with 60 second interval and six digit OTP
#       HOTP/T60/5      - HOTP time-based token with 60 second interval and five digit OTP
#       MOTP            - Mobile-OTP time-based token 10 second interval and six digit OTP
#       MOTP/E          - Mobile-OTP event-based token with six digit OTP
#
# For more info see: https://github.com/archiecobbs/mod-authn-otp/wiki/UsersFile
#

# Some users who have logged in at least once.

HOTP    barney        1234    8a2d55707a9084982649dadc04b426a06df19ab2 21      0 820658  2009-06-12T17:52:32L 192.168.1.1
HOTP    fred          5678    acbd18db4cc2f85cedef654fccc4a4d8bd537891 78      0 617363  2009-06-04T21:17:03L 192.168.1.2
HOTP/T  joe           999999  ef654fccdef654fccc4a4d8acbd18db4cc2f85ce -2      2 883913  2009-06-04T21:17:03L 10.1.1.153

# Wilma and Betty are new users. Note betty does not have a PIN so "-" is used instead as a placeholder

HOTP    wilma         5678    a4d8acbddef654fccc418db4cc2f85cea6339f00
HOTP    betty         -       54fccc418a4d8acbddef6db4cc2f85ce99321d64

# Here is a user who's PIN is verified externally using whatever "OTPAuthPINAuthProvider" list you have configured.
# E.g. to use an htpasswd type file, specify "OTPAuthPINAuthProvider file" and then "AuthUserFile /some/file".
HOTP    bambam        +       d8acbddef6db4cc254fccc418a4f85ce99321d64
To add users, add new lines to the file containing only the first four or five fields (see for example `wilma`, `betty`, and `bambam`). Some of the fields have been added to this file in newer **mod\_authn\_otp** versions. Newer versions of **mod\_authn\_otp** are backward compatible with older versions' file formats and will automatically upgrade them the first time they are used.

Locking

**mod\_authn\_otp** creates a lock file in the same directory as the users file in order to synchronize server threads so they don't try to update the users file at the same time. The file is empty; it just needs to be there so it can be locked.

Mobile-OTP

**mod\_authn\_otp** supports using the [Mobile-OTP](http://motp.sourceforge.net/) algorithm instead of HOTP/OATH. In this case the PIN (if any) should be entered into the token when generating the one-time password and _not_ as a prefix to the HTTP password. Use token type "MOTP" instead of "HOTP". Note: MOTP authentication is not compatible with the **OTPAuthPINAuthProvider** configuration directive. With MOTP tokens, the PIN must be explicitly provided in the users file.

Change permissions of the file.

chmod 0600 /etc/users.oath

Edit PAM

Edit /etc/pam.d/sshd to require TOTP for PAM authentication. Add after password-auth

auth    required   pam_oath.so usersfile=/etc/users.oath window=3 digits=6
Note: TOTP order Example for Debian 11. `window` is the search depth. If the `window` is 3, it allows TOTP codes generated up to +-1min from now. It allows the current code, previous and next. Needed when the time is not synced. If the server has 14:30:45 it allows TOTP codes generated from 14:30:00 to 14:31:29, that would make three different TOTP codes are valid at 14:30:45 for a `window` of 3. `digits` is the number of digits the code has. Depending on the order you want: Password then TOTP: `required` after `@include common-auth` 
@include common-auth
auth    required   pam_oath.so usersfile=/etc/users.oath window=3 digits=8
TOTP then Password: `required` before `@include common-auth` 
auth    required   pam_oath.so usersfile=/etc/users.oath window=3 digits=8
@include common-auth
TOTP only: `sufficient` before `@include common-auth` 
auth    sufficient   pam_oath.so usersfile=/etc/users.oath window=3 digits=8
@include common-auth
SSH public key is required for all of them. In theory `pam_oath.sh` should be added before or after `password-auth` but Debian does not have that line in the file so add if before or after `@include common-auth`.

Edit sshd_config

Edit /etc/ssh/sshd_config to require the TOTP code.

Authenticationmethods publickey,keyboard-interactive
PasswordAuthentication no
ChallengeResponseAuthentication yes

(Optional) do not require TOTP code for LAN adding the following lines to /etc/ssh/sshd_config.

Match Address 127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
  Authenticationmethods publickey

Restart sshd service unless you use a socket.

systemctl restart sshd

And you are good to connect using a TOTP code.

Generate TOTP code

With the seed 6b400f7b322b33a98b9ace4dbeff3e (hex) you want to generate a TOTP code.

Using oathtool:

oathtool --totp 6b400f7b322b33a98b9ace4dbeff3e --digits 8

You can also use keepasXC, pass or authenticator (gnome).

The code only works once, if you want to open two or more instances you have to wait for the code to regenerate.

Generate QR code

You need a base32 code for qrencode. You can get it with oathtool.

  • 6b400f7b322b33a98b9ace4dbeff3e seed in hex
  • NNAA66ZSFMZ2TC42ZZG357Z6 seed in base32

With the seed you generate the secret Base32 secret. This code is the password and must be treated as such.

oathtool -v --digits 8 6b400f7b322b33a98b9ace4dbeff3e
---
Hex secret: 6b400f7b322b33a98b9ace4dbeff3e
Base32 secret: NNAA66ZSFMZ2TC42ZZG357Z6
Digits: 8
Window size: 0
Start counter: 0x0 (0)

23340342

Or oathtool --verbose --totp 6b400f7b322b33a98b9ace4dbeff3e --digits=8 | grep Base32 | cut -d ' ' -f 3

And create an image.

qrencode -o user.png 'otpauth://totp/user@machine?secret=NNAA66ZSFMZ2TC42ZZG357Z6&digits=8'

The QR code will be at user.png and the comment of the key will be user@machine.

You can also generate the QR code and the seed with gen-oath-safe. Source

gen-oath-safe $USER totp

Configure SELinux

Source

SELinux by default does not allow this method because ssh can not write in /etc/. The official documentation does not cover this issue.

Install audit2allow in CentOS. Search for a package with yum provides \*/audit2allow

sudo yum install policycoreutils-python

Disable SELinux (permissive) to log the requests that ssh makes.

sudo setenforce 0

May need to try to log in to generate logs.

You can see the error and the reason with:

sudo cat /var/log/audit/audit.log | grep ssh | audit2why
---------------------------------------------------------
type=AVC msg=audit(1565369402.846:97): avc:  denied  { nnp_transition } for  pid=1584 comm="sh" scontext=system_u:system_r:init_t:s0 tcontext=system_u:system_r:iptables_t:s0 tclass=process2 permissive=0

    Was caused by:
        Missing type enforcement (TE) allow rule.

        You can use audit2allow to generate a loadable module to allow this access.

Create a policy for SELinux. This will list the rules that may fix the issue (they do) in order to revise them before creating a policy. I am sure there is a better, more secure way but this is the first time I used SELinux and this worked.

sudo grep sshd_t /var/log/audit/audit.log | audit2allow --module=sshdlocal
---------------------------------------------------------------------------
module sshdlocal 1.0;

require {
    type etc_t;
    type sshd_t;
    class file { create rename unlink write };
}

#============= sshd_t ==============

#!!!! WARNING: 'etc_t' is a base type.
allow sshd_t etc_t:file { create rename unlink write };

Create a custom policy module for SELinux.

sudo grep sshd_t /var/log/audit/audit.log | audit2allow --module-package=sshdlocal

Enable SELinux (enforcing) again.

sudo setenforce 1

Install the module.

sudo semodule --install=sshdlocal.pp

Make sure the policy is loaded.

sudo semodule --list-modules | grep sshdlocal

Enforcing SELinux may disable the policy (Don't know how true this is).

Certificates

  • The CA keys are just normal keys.
  • You can use different keys to sing user and host certificates. Multiple CAs.
  • A principal is a tag which can be used to grant permissions.
  • If the key has no principals it is not permitted to authenticate.
  • Check the ID and principals with ssh-keygen -Lf id_ed25519-cert.pub
  • If you implement this properly, a serial number -z must be used for key revoking.
  • Some recommend to not use any real username as a principal.
  • If a key has a principal which matches a username, that key will be allowed to log in as that username.
  • sudo is only needed to read the certificate. If your user can read the certificate sudo is not needed.

Create the CA key pair

sudo ssh-keygen -t ed25519 -f /etc/ssh/ca

Authenticate users

You need a key pair to be signed. For this example it is /home/user/.ssh/id_ed25519.pub

Sign a key pair

sudo ssh-keygen -s /etc/ssh/ca -I user -n management,read -O permit-port-forwarding -V "-1m:forever" /home/user/.ssh/id_ed25519.pub
  • -s path to CA key
  • -I ID of the key logged by the server when the certificate is used
  • -n principal of the key
  • -O key options. Refer to the man ssh-keygen page.
  • -V validity of the key.

man ssh-keygen

This process generates /home/user/.ssh/id_ed25519-cert.pub

Configure the server

Copy the CA's public key /etc/ssh/ca.pub to the server.

/etc/ssh/sshd_config
---
TrustedUserCAKeys /etc/ssh/ca.pub
PasswordAuthentication no
AuthorizedPrincipalsFile /etc/ssh/auth_principals/%u

Add principals to access a user. The principal management will be allowed to log in as logs and psql. The principal read will only be allowed to log in as logs.

echo -e "management\nread\n" | sudo tee -a /etc/ssh/auth_principals/logs
echo -e "management" | sudo tee -a /etc/ssh/auth_principals/psql

Restart the service

sudo service sshd restart

Connect

ssh 10.0.0.1 -l user -i /home/user/.ssh/id_25519

Authenticate hosts

You need a host key pair to sign. They are /etc/ssh/ssh_host*.

Sign the key

Copy /etc/ssh/ssh_host_ed25519_key.pub from the server to sign it locally, sign it and upload it back to the server.

sudo ssh-keygen -s /etc/ssh/ca -I hostname -h -n host.domain.local,192.168.0.34 -V "-1m:forever" ssh_host_ed25519_key.pub
  • -s path to CA key
  • -I ID of the key
  • -h make a host key instead of a user key.
  • -n principal(s) of the key. Allowed hostnames.
  • -V validity of the key.

Configure the server

Edit /etc/ssh/sshd_config

HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub

Configure the client

Add the CA public key to ~/.ssh/known_hosts to allow pricipals domain.local and 192.168.1.0/24:

@cert-authority *.domain.local,192.168.1.* ssh-ed25519 AAAA...

Or any pricipal:

@cert-authority * ssh-ed25519 AAAA...

*.domain.local and 192.168.1.* are the principal names that the SSH host key has been signed with (cert). * is a wildcard.

This principals are the name (ssh host.domain.local) you use to connect. They can be: - An IP address (192.168.1.23) - An entry in /etc/hosts (192.168.1.23 host.domain.local) - A DNS resolved name

Connect

Remove previously cache identities.

ssh-keygen -R 10.0.0.1

Log in

ssh 10.0.0.1 -l user -i /home/user/.ssh/id_25519

You will not see The authenticity of host '10.0.0.1 (10.0.0.1)' can't be established.

Single Packet Authorization SPA

TBD

Fail2ban

WIP

Install

sudo paman -S fail2ban

Default config at /etc/fail2ban/jail.conf. This config may be overwritten, it is reccomended to create /etc/fail2ban/jail.local.

Configure fail2ban at /etc/fail2ban/jail.local.

[DEFAULT]
bantime   = 86400 ## 1 day
findtime  = 120
maxretry  = 2
logtarget = /var/log/fail2ban/fail2ban.log

Create the logging directory as root.

sudo mkdir /var/log/fail2ban

Configure a custom jail for ssh at /etc/fail2ban/jail.d/sshd.local.

[sshd]
enabled   = true
filter    = sshd
banaction = iptables
backend   = systemd
maxretry  = 5
#findtime  = 86400 ## 1 day
bantime   = 1209600 ## 2 weeks
ignoreip  = 127.0.0.1/8 192.168.1.1/24 10.0.0.5

Hardening the fail2ban service at /etc/systemd/system/fail2ban.service.d/override.conf. Make the directory first sudo mkdir /etc/systemd/system/fail2ban.service.d/.

[Service]
PrivateDevices=yes
PrivateTmp=yes
ProtectHome=read-only
ProtectSystem=strict
NoNewPrivileges=yes
ReadWritePaths=-/var/run/fail2ban
ReadWritePaths=-/var/lib/fail2ban
ReadWritePaths=-/var/log/fail2ban
ReadWritePaths=-/var/spool/postfix/maildrop
CapabilityBoundingSet=CAP_AUDIT_READ CAP_DAC_READ_SEARCH CAP_NET_ADMIN CAP_NET_RAW

Start and enable the service.

sudo systemctl daemon-reload
sudo systemctl start fail2ban
sudo systemctl enable fail2ban

Check the status.

sudo fail2ban-client status sshd
---
Status for the jail: sshd
|- Filter
|  |- Currently failed: 1
|  |- Total failed: 8
|  `- Journal matches:  _SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
   |- Currently banned: 1
   |- Total banned: 1
   `- Banned IP list:   192.168.100.1

Unban an IP

sudo fail2ban-client set sshd unbanip 192.168.100.1

Escape characters

Escape characters only work after a new line.

  • ~. Disconnect.
  • ~^Z Background ssh.
  • ~# List forwarded connections.
  • ~& Background ssh at logout when waiting for forwarded connection / X11 sessions to terminate.
  • ~? Display a list of escape characters.
  • ~B Send a BREAK to the remote system (only useful if the peer supports it).
  • ~C Open command line (ssh console). Currently this allows the addition of port forwardings using the -L, -R and -D options.
  • ~R Request rekeying of the connection (only useful if the peer supports it).
  • ~V Decrease the verbosity (LogLevel) when errors are being written to stderr.
  • ~v Increase the verbosity (LogLevel) when errors are being written to stderr.

SSH console options.

ssh> -L 9001:127.0.0.1:80
ssh> -KL 9001
ssh> -R 10.0.0.1:22:127.0.0.1:9001
ssh> -KR 10.0.0.1:22
ssh> -h

In an anidated session, like a bastion host (A (client) -> B (bastion) -> C (server)), pressing ~ once will control the bastion session (A -> B) while pressing ~ twice will control the remote host (B -> C).

Rate limit

WIP

$IPT -A INPUT  -i ${inet_if} -p tcp --dport ${ssh_port} -m state --state NEW -m limit --limit 3/min --limit-burst 3 -j ACCEPT

TCP wrappers

This is an old way to restrict access without a firewall.

Chroot Jails

WIP

Audit SSH

ssh-audit

Download

git clone https://github.com/jtesta/ssh-audit

Execute

python ssh-audit.py server.local

Results

[...]
# algorithm recommendations (for OpenSSH 8.0)
(rec) -diffie-hellman-group-exchange-sha256 -- kex algorithm to remove
(rec) +diffie-hellman-group14-sha256        -- kex algorithm to append
(rec) +diffie-hellman-group16-sha512        -- kex algorithm to append
(rec) +diffie-hellman-group18-sha512        -- kex algorithm to append
(rec) -hmac-sha2-256                        -- mac algorithm to remove
(rec) -hmac-sha2-512                        -- mac algorithm to remove
(rec) -umac-128@openssh.com                 -- mac algorithm to remove

Transform a PuTTY key to an OpenSSH one

puttygen private.ppk -O private-openssh -o private.ssh

Source

Logs

WIP

sudo sshd -T ## test the validity of the keys sudo sshd -t