How to setup python in Dreamhost’s shared hosting

#python #dreamhost #guide #gist

This article is originally stored as a github gist.

I use Dreamhost shared hosting and everytime I wanto to setup a python website/api/whatever, I struggle crawling over a number of pages for installation instructions and troubleshooting, so I decided to compile myself a guide to get it done properly from scratch.

This manual is in development. I will write it as I go through issues.

Enable Passenger in the Dreamhost Shared domain

https://help.dreamhost.com/hc/en-us/articles/216385637-How-do-I-enable-Passenger-on-my-domain-

Users, Files, and Paths -> Web directory

Set this option to something like applicationname/public.

  • The files on the public folder will be served by the webserver as assets (Images, JS, CSS, even PHP files).
  • If no file is found, the request will be routes to the python application.
  • Passenger (Dreamhost’s webserver) will look for a file passenger_wsgi.py on the parent folder applicationname

Web Options -> Passenger (Ruby/NodeJS/Python apps only)

Enable this option.

Directories to store source code and binaries

We will need to download build some stuff from the source code, so we will need directories to organise them the right way:

1
2
3
4
5

# Directory for source code (where binaries will be compiled from)
mkdir ~/src

# Directory for binaries
mkdir ~/opt

OpenSSL

We will work with a newer version of Python and it requires an updated openssl library.

Reference: https://help.dreamhost.com/hc/en-us/articles/360001435926-Installing-OpenSSL-locally-under-your-username

Download and build the source

1
2
3
4
5
6
7

cd ~/src
wget https://www.openssl.org/source/openssl-1.1.1b.tar.gz
tar zxvf openssl-1.1.1b.tar.gz
cd ~/src/openssl-1.1.1b
./config --prefix=$(readlink -m ~)/opt/openssl --openssldir=$(readlink -m ~)/opt/openssl no-ssl2
make
make test

Remember to check the openssl package integrity before installing it, by comparing the output of these commands:

  • curl https://www.openssl.org/source/openssl-1.1.1b.tar.gz.sha256
  • sha256sum openssl-1.1.1b.tar.gz

Ensure you got a result All tests successful from make test before proceeding with the installation.

Install OpenSSL

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# install openssl to ~/opt/openssl
cd ~/src/openssl
make install

# add openssl flags and reference to .bash_openssl
cat <<EOF > ~/.bash_openssl
export PATH=$(readlink -m ~)/opt/openssl/bin:\$PATH
export LD_LIBRARY_PATH=$(readlink -m ~)/opt/openssl/lib
export LC_ALL="en_US.UTF-8"
export LDFLAGS="-L$(readlink -m ~)/opt/openssl/lib -Wl,-rpath,$(readlink -m ~)/opt/openssl/lib"
EOF

# source .bash_openssl into .bash_profile
echo -e "\n# enables the local installation of openssl from ~/opt/openssl\n# (comment the line below to disable)\nsource ~/.bash_openssl\n" >> ~/.bash_profile

# enables the custom openssl in the current shell without require to log out and log back in
. ~/.bash_openssl

Check installation

Check if openssl was correctly installed:

1
2

which openssl # expected: /home/USERNAME/opt/openssl/bin/openssl
openssl version # expected: OpenSSL 1.1.1b  26 Feb 2019

If failed and your shell session started before the installation, run . ~/.bash_openssl and check again

Python 3.8 (with SSL support)

https://help.dreamhost.com/hc/en-us/articles/115000702772-Installing-a-custom-version-of-Python-3

Download and build the source

Download

1
2
3
4

cd ~/src
wget --no-check-certificate https://www.python.org/ftp/python/3.8.1/Python-3.8.1.tgz
tar zxvf Python-3.8.1.tgz 
cd ~/src/Python-3.8.1

Enable compiler flags for SSL support

Edit the file ~/src/Python-3.8.1/Modules/Setup, uncomment and edit the following part (Line 205):

Remember to replace USERNAME accordingly

1
2
3
4
5
6
7
8
9

# Socket module helper for socket(2)
_socket socketmodule.c

# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
SSL=/home/USERNAME/opt/openssl
_ssl _ssl.c \
   -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
   -L$(SSL)/lib -lssl -lcrypto

Build

If you intend to use the config flag –enable-optimizations, check what the page https://help.dreamhost.com/hc/en-us/articles/115000702772-Installing-a-custom-version-of-Python-3 says about it (you have to contact support to disable some memory limits during the compilation and etc, so just go for it if you really need. The current ubuntu version can be retrieved with lsb_release -a

1
2
3

cd ~/src/Python-3.8.1
./configure --prefix=$(readlink -m ~)/opt/python-3.8.1
make

Install Python

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# installs python to ~/opt/python-3.8.1
cd ~/src/Python-3.8.1
make install

# set python as default interpreter at .bash_python
cat <<EOF > ~/.bash_python
export PATH=$(readlink -m ~)/opt/python-3.8.1/bin:\$PATH
EOF

# source .bash_python into .bash_profile
echo -e "\n# enables the local installation of python from ~/opt/python-3.8.1\n# (comment the line below to disable)\nsource ~/.bash_python\n" >> ~/.bash_profile

# enables the custom openssl in the current shell without require to log out and log back in
. ~/.bash_python

Check Python installation

Check if Python (and PIP) are correctly installed:

1
2
3

which python3 # expected: /home/USERNAME/opt/python-3.8.1/bin/python3
python3 --version # expected: Python 3.8.1
which pip3 # expected: /home/USERNAME/opt/python-3.8.1/bin/pip3"

If failed and your shell session started before the installation, run . ~/.bash_python and check again

Upgrde PIP to the latest version

1

pip3 install --upgrade pip

Virtualenv

https://help.dreamhost.com/hc/en-us/articles/115000695551

Install virtualenv for Python3

1

pip3 install virtualenv

Check installation

which virtualenv # expected: /home/USERNAME/opt/python-3.8.1/bin/virtualenv

Project setup

Create a virtualenv for the project

  1. Go to the project folder (one level above the public folder set in Dreamhost domain’s option Web directory).
  2. Run virtualenv --python=$(readlink -m ~)/opt/python-3.8.1/bin/python3.8 venv

venv at the end is a folder that will be created to store binaries and libraries. You can use another name, as well as add it to .gitignore to save space in the repository.

Check if virtualenv is working properly

Run . venv/bin/activate (replacing venv with whatever you used for the virtualenv name.

  1. Your shell prompt should be prefixed with a (venv).
  2. which python and which pip should point to binaries inside the venv folder
  3. python --version should return Python 3.8.1

Using virtualenv

All command below must be executed from the project application root folder, where the virtualenv folder is located. venv here can be replaced with whatever you used for your virtualven name.

Action Command
Activate . venv/bin/activate
Deactivate deactivate
Remember packages pip freeze > requirements.txt
Install packages pip install -r requirements.txt

It is handy to have a symlink to the activate command in the application root

# run in the project root directory
ln -s $(readlink -m venv/bin/activate) activate

A Hello World WSGI application using the local instance of Python3

Create a file passenger_wsgi.py in the application root (one level about the public Web Directory) with the following:

import sys,os

app_root_dir = os.path.dirname(os.path.realpath(__file__))
INTERP = os.path.join(app_root_dir, 'venv/bin/python')
if sys.executable != INTERP:
    os.execl(INTERP, INTERP, *sys.argv)
sys.path.insert(0,app_root_dir)

def application(environ, start_response):
    start_response('200 OK', [('Content-type', 'text/plain')])
    return ["Hello, world!"]

Also, create a file tmp/restart.txt from the application root. Everytime you run touch tmp/restart.txt, the application will restart. This have to be done to see changes.

Error Handling

Enable code 500 errors in the browser

Add the following to public/.htaccess:

PassengerFriendlyErrorPages on

Not ideal for production environments, use only for dev.

More links