Torifying a Static Web Site with Docker and Ubuntu 20

This documents the easiest way to Tor-ify a static website with Docker. These instructions were executed on a default Ubuntu 20 Server installation.

The Website

Navigate to your preferred working directory and execute the following:

1
mkdir -p tor-site/www && cd tor-site

docker-compose.yml

Create a file called docker-compose.yml and paste:

1
2
3
4
5
6
7
8
9
10
11
12
13
version: "3.9"
services:
  nginx:
    image: nginx
    restart: unless-stopped
    expose:
      - "80"
    volumes:
      - ./www:/usr/share/nginx/html
networks:
  default:
    external:
      name: tor-proxy_default

This sets up the nginx container, which will serve the static website. The website itself will go into the www/ directory. Create a new file called index.html inside the www directory:

1
touch www/index.html

Paste this into www/index.html:

1
2
3
4
5
6
7
8
<html>
  <head>  
    <title>Bonjour</title>
  </head>
  <body>
    <h1>Bienvenue sur le darkweb</h1>     
  </body>   
</html>

Deploy the Server:

If you’re familiar with Docker, you will have noticed a default network called tor-proxy_default. This doesn’t exist yet. You only need to execute the following one time:

1
docker network create tor-proxy_default

Now, your website should be ready to go.

1
docker compose up -d

The Tor Proxy

Navigate back to your preferred working directory:

1
cd ..

And create a new directory alongside your tor-site/:

1
mkdir -p tor-proxy/config && cd tor-proxy

This is where you set up the Tor proxy itself. It is in a seperate directory from where you’ve set up your static nginx website.

Dockerfile

Create a file called Dockerfile and paste:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
FROM debian
ENV NPM_CONFIG_LOGLEVEL warn

ENV DEBIAN_FRONTEND noninteractive

EXPOSE 9050                    
      
RUN apt-get update

# `apt-utils` squelches a configuration warning
# `gnupg` is required for adding the `apt` key
RUN apt-get -y install apt-utils wget gnupg apt-transport-https
    
#
# Info from: https://packages.debian.org/stretch/amd64/libevent-2.0-5/download
#
RUN echo "deb http://ftp.ca.debian.org/debian stretch main" | tee -a /etc/apt/sources.list.d/torproject.list

#
# From https://support.torproject.org/apt/
#
RUN echo "deb [signed-by=/usr/share/keyrings/tor-archive-keyring.gpg] https://deb.torproject.org/torproject.org stretch main" | tee -a /etc/apt/sources.list.d/torproject.list
RUN echo "deb-src [signed-by=/usr/share/keyrings/tor-archive-keyring.gpg] https://deb.torproject.org/torproject.org stretch main" | tee -a /etc/apt/sources.list.d/torproject.list
RUN wget -qO- https://deb.torproject.org/torproject.org/A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89.asc | gpg --dearmor | tee /usr/share/keyrings/tor-archive-keyring.gpg >/dev/null
RUN apt-get update
RUN apt-get -y upgrade
RUN apt-get -y install tor deb.torproject.org-keyring

# The debian image does not create a default user
RUN useradd -m user
USER user

# Run the Tor service
CMD /usr/bin/tor -f /etc/tor/torrc

This creates the Docker image.

docker-compose.yml

I think it’s best to manage the created image with docker-compose. Create a new file called docker-compose.yml and paste:

1
2
3
4
5
6
7
version: "3.9"                 
services:
  tor:
    build: .
    restart: unless-stopped
    volumes:
      - ./config/torrc:/etc/tor/torrc

Configure the Tor Proxy

You can host several websites or web applications through a Tor proxy. Notice the shared volume declared in the docker-compose.yml file. You need to create a file called ./config/torrc:

1
touch config/torrc

This is read from inside the Tor Docker container. Paste the following into ./config/torrc:

1
2
3
HiddenServiceDir /home/user/.tor/hidden_app_1/
HiddenServicePort 80 tor-site-nginx-1:80
HiddenServiceVersion 3

Deploy the Tor Proxy

1
docker compose up -d

You can find your .onion hostnames in the .tor directory in the container:

1
docker compose exec tor cat /home/user/.tor/hidden_app_1/hostname

You’ll see an address that looks like this:

1
257472yzjwmkcts7mmp2nl4v32brv5gq7ybx2ac6luv6ddjnipwv2xyd.onion

Navigate to your Torified website by entering it into your Tor-enabled browser.

Network Troubleshooting

Whoa! I had never run into this issue before…

I discovered a frustrating error that took some effort to solve. Basically, when it comes to sharing networking with the host, a Docker image defaults to an MTU of 1500. The network adapter on my server has an MTU of 1280. The symptom of this was that the Tor Docker container couldn’t run apt-get because the connection would always time out.

This is the resource that solved the problem: https://www.civo.com/learn/fixing-networking-for-docker

To determine if you have this issue, execute the following in the tor-proxy directory:

1
ip a

You will see something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# ...

2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1280 qdisc mq state UP group default qlen 1000
    link/ether 74:d4:35:e9:5b:7e brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.8/24 brd 192.168.2.255 scope global enp2s0
       valid_lft forever preferred_lft forever
    inet6 fe80::76d4:35ff:fee9:5b7e/64 scope link 
       valid_lft forever preferred_lft forever

# ...


5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:a3:b9:76:b0 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:a3ff:feb9:76b0/64 scope link 
       valid_lft forever preferred_lft foreve

The values associated with mtu need to match. This resource explains how you might fix the problem.


Plutus Playground Minimal Ubuntu Setup

This document is written for the third iteration of Cardano’s Plutus Pioneers Program. It demonstrates the steps required to install and execute the Plutus Playground Server.

All credit belongs to Deceptikon, who can be reached on Discord: @Deceptikon#9964. I am simply relaying his excellent work here.

Install nix:

1
sh <(curl -L https://nixos.org/nix/install) --no-daemon

Close your active terminal and reopen it. Doing this provides a fresh terminal session with nix enabled (p.s., I am brand new to nix).

nix requires some configuration:

1
2
3
cd ~/.config
mkdir nix
cd nix

Create a new nix.conf file. I use vim for most things. You can use whatever you like.

1
vim nix.conf

Paste the following lines into the nix.conf:

1
2
3
substituters = https://hydra.iohk.io https://iohk.cachix.org https://cache.nixos.org/
trusted-public-keys = hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= iohk.cachix.org-1:DpRUyj7h7V830dp/i6Nti+NEO2/nhblbov/8MW7Rqoo= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
experimental-features = nix-command

Save and close the document.

Test to see if nix is installed:

1
nix-env --version

You should see something like this:

1
nix-env (Nix) 2.5.1

Now it’s time to download and build the Plutus Playground. It’s up to you where you store the software, just so long as you are mindful of its location. For demonstration purposes, the relevant plutus-app software will be located in your home directory:

1
2
3
cd
git clone https://github.com/input-output-hk/plutus-apps
cd plutus-apps

At the time of writing, you need to checkout the correct revision of the repository you just downloaded. You can determine the revision number by consulting the plutus-pioneer-program repository.

Using the code for Week 1 as an example, look for the code/week01/cabal.project file. You will find the required revision number under the source-repository-package > tag configuration. I suspect you will need to look for this revision in each week’s code.

1
git checkout 41149926c108c71831cfe8d244c83b0ee4bf5c8a

Finally, build the client and server software.

1
nix-build -A plutus-playground.server

This may take some time. Deceptikon has some good advice:

If this step takes longer than 10 minutes, abort! If the build takes longer than that, the IOHK binary caches are not configured correctly. Most likely your nix.conf is not in the correct folder. It should be right if you EXACTLY followed the steps and are using a fresh install of Ubuntu. Others such as MacOS may require nix.conf to be in a different folder or may require multi-user install of nix. If you think to yourself, as I did, “I’ll just let it run overnight!” the only result will be a huge waste of time on something that doesn’t work

I’m not sure if the following is totally necessary, but Deceptikon recommends you close the working terminal window. Open it again and execute: 

1
. /home/yourusernamehere/.nix-profile/etc/profile.d/nix.sh

Be sure to change the yourusernamehere part of the path to your actual username.

Go to your plutus-apps directory (if you’re not there already) and launch the nix shell:

1
2
cd ~/plutus-apps
nix-shell

This will probably take a bit of time on first execution.

Once nix-shell is all warmed up:

1
2
cd plutus-playground-server/
plutus-playground-server

This is what I saw on my first successful execution:

1
2
3
4
5
6
7
8
plutus-playground-server: for development use only
[Info] Running: (Nothing,Webserver {_port = 8080, _maxInterpretationTime = 80s})
Initializing Context
Initializing Context
Warning: GITHUB_CLIENT_ID not set
Warning: GITHUB_CLIENT_SECRET not set
Warning: JWT_SIGNATURE not set
Interpreter ready

Look for the Interpreter ready. That’s the important part!

At this point you have a server running. Open a second terminal window and ensure you are in the plutus-apps directory. You need to start nix-shell again:

1
2
3
4
cd ~/plutus-apps
nix-shell
cd plutus-playground-client
npm run start

As before, if this the first time executing the program, it may take a while to get going.

Once ready, you can open the Playground interface in your favourite browser at https://localhost:8009/. You may get a scary Invalid Certificate error, but you can safely ignore that.

Now try compiling some of the sample code: Lecture #1 - Part #5 - Auction Contract on the Playground…

All good?

Optional: For convenience, build and serve the Plutus documentation locally:

1
2
3
cd ~/workspace/plutus-apps
nix-shell
build-and-serve-docs

View at http://localhost:8002/haddock


Plutus Pioneers - Weekly Environment Setup with git and nix-shell

Week 2 Lecture 2 is upon us, Plutus Pioneers! In order to follow along, you’ll need to update your work environment. You’ll likely need to follow these steps for each subsequent week.

Checkout this Week’s Revision

Last week you cloned and built the plutus-apps repository on your computer. Consider the plutus-apps directory your launch point for coursework. It was within this context that you built and executed the Plutus Playground Server and its client.

This first step is a little hairy. Hang on tight…

Judging by the steps taken in Week 1, you’ll likely need to rebuild the Playground client and server every new week. To do this, you must checkout the correct revision of the plutus-apps repository. You can determine the revision number by consulting the plutus-pioneer-program repository.

Using the code for Week 2 as an example, look for the code/week02/cabal.project file. You will find the required revision number under the source-repository-package > tag configuration. Week 2’s revision number looks like this: 6aff97d596ac9d59460aab5c65627b1c8c0a1528.

Get ready to copy/paste that number. But first, navigate to the plutus-apps directory. For me, this step looks like this:

1
cd ~/workspace/plutus-apps

Update the repository in case there were any recent changes:

1
2
git checkout main
git pull

Now, checkout the correct revision:

1
git checkout 6aff97d596ac9d59460aab5c65627b1c8c0a1528

Phew! I don’t know for sure, but I suspect you’ll need to do this every week.

Rebuild and Execute the Plutus Playground

This assumes that you were successful in setting up nix.

Continuing from above, execute:

1
nix-build -A plutus-playground.server

Start nix-shell:

1
nix-shell

Start the Plutus Playground Server:

1
2
cd plutus-playground-server/
plutus-playground-server

The server should now be executing in the foreground of your terminal. Leave this open. Open a second terminal window and start the client (steps consolidated here):

1
2
3
4
cd ~/workspace/plutus-apps/
nix-shell
cd plutus-playground-client
npm run start

Now the client is running. You can see it at https://localhost:8009 in your favourite browser. Leave this terminal open as well.

Update the Course Code and Get to Work

Open a new terminal window and start nix-shell:

1
2
cd ~/workspace/plutus-apps/
nix-shell

By now, it should be clear that the plutus-apps directory contains what you need to get the provided environment up and running. Now, navigate to your plutus-pioneers-program directory:

1
cd ../plutus-pioneers-program

Update the local repository:

1
git pull

You now have this week’s lesson material. Navigate to the directory and build the project:

1
2
cd code/week02/
cabal build

Time for me to start watching the new lectures…


Minimal Haskell Dev Setup on Ubuntu

This documents the steps required to prepare a development environment for Haskell in Ubuntu. All of this is straight-up jacked from here. I’m brand new to the Haskell ecosystem. I am learning the language so that I can build and test Plutus smart contracts on Cardano.

Note to Plutus Pioneers: the following instructions are not executed from within the context of nix (i.e., I’m not running nix when I’m running these commands…)

Basic tool setup

First, install some system dependencies:

1
sudo apt install build-essential curl libffi-dev libffi6 libgmp-dev libgmp10 libncurses-dev libncurses5 libtinfo5

Then, install GHCup:

1
curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh

You will be prompted to make some decisions. I generally like to stick with defaults, and that’s mostly true here. However,

  1. I answered Y to installing the haskell-language-server. This is so I can leverage language features in my favourite editor, vim.
  2. I answered Y to installing stack. Here’s some context.

At the end of it, you should have GHC, cabal-install, stack, and haskell-language-server.

In all the steps taken so far I’m not sure what went wrong, but stack never actually arrived. Thankfully, the Haskell docs provide a backup:

1
curl -sSL https://get.haskellstack.org/ | sh

Optional: Configure the Language Server for Vim

vim is the only editor I use. If you have the haskell-language-server installed, you can do something similar with your favourite tool.

I tried to take the path of least resistance by using the Coc configuration/installation instructions. Again, I’ve purposefully avoided executing within the provided Plutus nix-shell environment. The instructions are distilled as follows…

Install vim-plug:

1
2
curl -fLo ~/.vim/autoload/plug.vim --create-dirs \
    https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim

Configure vim-plug:

1
vim ~/.vimrc

Paste this somewhere toward the top of the file where you won’t be busting up any logical blocks:

1
2
3
4
5
6
7
"""
" vim-plug extensions
call plug#begin('~/.vim/plugged')

Plug 'neoclide/coc.nvim', {'branch': 'release'}

call plug#end()

Run this command in vim:

1
:PlugInstall

This will download whatever plugins are included before the plug#begin and plug#end.

Again, issue this command in vim:

1
:CocConfig

Paste this into the Coc config file:

1
2
3
4
5
6
7
8
9
10
{
  "languageserver": {
    "haskell": {
      "command": "haskell-language-server-wrapper",
      "args": ["--lsp"],
      "rootPatterns": ["*.cabal", "stack.yaml", "cabal.project", "package.yaml", "hie.yaml"],
      "filetypes": ["haskell", "lhaskell"]
    }
  }
}

Bootstrap a project with Stack

I spend most of my time in a node environment. If you’re familiar with the Javascript ecosystem, stack appears analagous to npm.

To start a new project with the default template, navigate to your preferred project directory and initialize:

1
2
3
cd ~/workspace
stack new my-project 
cd my-project

You now have a directory containing almost everything you need to start work. Facilities for testing are sketched in but aren’t actually configured when bootstrapping from the default template.

Just for fun, try:

1
stack test

You can also build the minimal project:

1
stack build

And execute:

1
stack exec my-project-exe

To launch the REPL:

1
stack ghci

Haskell and all it’s goodies, ready to go.

Next up… I’m figuring out testing tools.


How to get started testing with Express, Jasmine, and Zombie

Okay, so I’m a little old fashioned. I’m also lazy and reluctant to learn new things. My buddy Dawson asked How do I get started with testing my web software? I said, start with node/express…

Express

Express is a web framework. You build server-side software with Express. Let’s bootstrap a project with express-generator:

1
npx express-generator --view=ejs myapp

This creates a skeleton application from which to launch development. The ejs stands for Embedded Javascript. To install:

1
2
cd myapp
npm install

Execute the server:

1
npm start

Assuming all is well, you can navigate to http://localhost:3000 to see your new app in action. Halt server execution by pressing Ctrl-C.

That’s great and all, but let’s get to testing…

Jasmine

I’m not sure Jasmine is trendy, but I’ve been using it for years. It takes minimal setup and can be neatly structured. It’s a good tool and the tests you write are easily adapted for execution by other test frameworks (if you decide you don’t like jasmine).

Add jasmine to your web application:

1
npm install --save-dev jasmine

jasmine is now a development dependency. Execute cat package.json to get a peak at how node manages its dependencies.

Initialize jasmine like this:

1
npx jasmine init

Most people like to script test execution with npm. Open the package.json file just mentioned. Configure "scripts": { "test": "jasmine" }… that is, make package.json look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
{
  "name": "myapp",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "start": "node ./bin/www",
    "test": "jasmine"
  },
  "dependencies": {
    "cookie-parser": "~1.4.4",
    "debug": "~2.6.9",
    "ejs": "~2.6.1",
    "express": "~4.16.1",
    "http-errors": "~1.6.3",
    "morgan": "~1.9.1"
  },
  "devDependencies": {
    "jasmine": "^3.5.0"
  }
}

If you do this correctly, you can run the following:

1
npm test

And expect to see something like this:

1
2
3
4
5
6
7
8
9
10
11
12
> myapp@0.0.0 test /home/daniel/workspace/myapp
> jasmine

Randomized with seed 44076
Started


No specs found
Finished in 0.003 seconds
Incomplete: No specs found
Randomized with seed 44076 (jasmine --random=true --seed=44076)
npm ERR! Test failed.  See above for more details.

jasmine is telling us that the tests failed because we have yet to write any. Being as lazy as I am, I try to find opportunities to skip unit tests and go directly to testing user behaviour. For this, I use a headless browser called Zombie.

Zombie

Like jasmine, I’m not sure zombie is the trendiest option out there, but I’ve always managed to get the two to play nicely together and have yet to find any serious shortcoming. Add zombie to your project like this:

1
npm install --save-dev zombie

Now we’re ready to write some tests…

Testing!

Oh wait, what’s this app supposed to do? Ummmmm…

For now, I’ll keep it really simple until my buddy Dawson comes up with tougher testing questions.

Purpose

I want to be able to load my app in a browser, enter my name in an input field, press Submit, and receive a friendly greeting in return.

When starting a new web application, the first test I write ensures my page actually loads in the zombie browser. Create a spec file:

1
touch spec/indexSpec.js

I use the word index in the sense that it’s the default first page you land on at any website. Test files end with the *Spec.js suffix by default. Execute cat spec/support/jasmine.json to see how jasmine decides which files to execute.

Open spec/indexSpec.js in your favourite editor and paste this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
const Browser = require('zombie');

Browser.localhost('example.com', 3000);

describe('the landing page', () => {

  let browser;

  /**
   * This loads the running web application
   * with a new `zombie` browser before each test.
   */
  beforeEach(done => {
    browser = new Browser();

    browser.visit('/', err => {
      if (err) return done.fail(err);
      done();
    });
  });

  /**
   * Your first test!
   *
   * `zombie` has loaded and rendered the page
   * returned by your application. Use `jasmine`
   * and `zombie` to ensure it's doing what you
   * expect.
   *
   * In this case, I just want to make sure a 
   * page title is displayed.
   */
  it('displays the page title', () => {
    browser.assert.text('h1', 'Friendly Greeting Generator'); 
  });

  /**
   * Put future tests here...
   */
  
  // ...
});

Simple enough. At this point you might be tempted to go make the test pass. Instead, execute the following to make sure it fails:

1
npm test

Whoa! What happened? You probably see something like this:

1
2
3
4
5
6
7
8
9
10
11
Randomized with seed 73862
Started
F

Failures:
1) the landing page displays the page title
  Message:
    Failed: connect ECONNREFUSED 127.0.0.1:3000
  Stack:

    ...

It’s good that it failed, because that’s an important step, but if you look closely at the error, connect ECONNREFUSED 127.0.0.1:3000 tells you your app isn’t even running. You’ll need to open another shell or process and execute:

1
npm start

Your app is now running and zombie can now send a request and expect to receive your landing page. In another shell (so that your app can keep running), execute the tests again:

1
npm test

If it fails (as expected), you will see something like this:

1
2
3
4
5
6
7
8
9
10
Randomized with seed 38606
Started
F

Failures:
1) the landing page displays the page title
  Message:
    AssertionError [ERR_ASSERTION]: 'Express' deepEqual 'The Friendly Greeting Generator'
  Stack:
    error properties: Object({ generatedMessage: true, code: 'ERR_ASSERTION', actual: 'Express', expected: 'Friendly Greeting Generator', operator: 'deepEqual' })

That’s much better. Now, having ensured the test fails, make the test pass. Open routes/index.js in your project folder and make it look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
var express = require('express');
var router = express.Router();

/* GET home page. */
router.get('/', function(req, res, next) {
  // The old command
  //res.render('index', { title: 'Express' });

  // The new test-friendly command
  res.render('index', { title: 'The Friendly Greeting Generator' });
});

module.exports = router;

Execute the tests again:

1
npm test

And you will see:

1
2
3
4
5
6
7
8
9
Randomized with seed 29903
Started
F

Failures:
1) the landing page displays the page title
  Message:
    AssertionError [ERR_ASSERTION]: 'Express' deepEqual 'The Friendly Greeting Generator'
  Stack:

Oh no! Not again! Go back and check… yup, you definitely changed the name of the app. What could be wrong?

You need to restart your server in your other shell. Exit with Ctrl-C and restart with npm start. (Yes, there is a much better way of doing this).

Having restarted your application, execute the tests again with npm test. You will see this:

1
2
3
4
5
6
7
8
Randomized with seed 46658
Started
.


1 spec, 0 failures
Finished in 0.071 seconds
Randomized with seed 46658 (jasmine --random=true --seed=46658

Awesome. Your first test passes. Recall the stated purpose of this app:

I want to be able to load my app in a browser, enter my name in an input field, press Submit, and receive a friendly greeting in return.

Using this user story as a guide, you can proceed writing your tests. So far, the first part of the story has been covered (i.e., I want to be able to load my app in a browser). Now to test the rest…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
  // ...
  // Add these below our first test in `indexSpec.js`

  it('renders an input form', () => {
    browser.assert.element('input[type=text]'); 
    browser.assert.element('input[type=submit]'); 
  });

  it('returns a friendly greeting if you enter your name and press Submit', done => {
    browser.fill('name', 'Dan');
    browser.pressButton('Submit', () => {
      browser.assert.text('h3', 'What up, Dan?');
      done();
    });
  });

  it('trims excess whitespace from the name submitted', done => {
    browser.fill('name', '                Dawson               ');
    browser.pressButton('Submit', () => {
      browser.assert.text('h3', 'What up, Dan?');
      done();
    });
  });


  it('gets snarky if you forget to enter your name before pressing Submit', done => {
    browser.fill('name', '');
    browser.pressButton('Submit', () => {
      browser.assert.text('h3', 'Whatevs...');
      done();
    });
  });

  it('gets snarky if you forget to enter a blank name before pressing Submit', done => {
    browser.fill('name', '         ');
    browser.pressButton('Submit', () => {
      browser.assert.text('h3', 'Please don\'t waste my time');
      done();
    });
  });
});

You can push this as far as you want. For example, you might want to ensure your audience doesn’t enter a number or special characters for a name. The ones above define the minimal test-coverage requirement in this case.

Make sure these new tests fail by executing npm test. You won’t need to restart the server until you make changes to your app (yes, you should find a better way to manage this).

Make the tests pass

You should try doing this yourself before you skip ahead. I’ll give you a couple of clues and then provide the spoiler. In order to get these tests to pass, you’ll need to add a route to routes/index.js and you’ll need to modify the document structure in views/index.ejs.

Did you try it yourself?

Here’s one possible answer:

Make routes/index.js look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
var express = require('express');
var router = express.Router();

/* GET home page. */
router.get('/', function(req, res, next) {
  res.render('index', { title: 'The Friendly Greeting Generator', message: '' });
});

router.post('/', function(req, res, next) {
    
  let message = 'Whatevs...'
  if (req.body.name.length) {

    let name = req.body.name.trim();

    if (!name.length) {
      message = 'Please don\'t waste my time';
    }
    else {
      message = `What up, ${name}?`;
    }
  }

  res.render('index', { title: 'The Friendly Greeting Generator', message: message });
});


module.exports = router;

Make views/index.js look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<!DOCTYPE html>
<html>
  <head>
    <title><%= title %></title>
    <link rel='stylesheet' href='/stylesheets/style.css' />
  </head>
  <body>
    <h1><%= title %></h1>
    <p>Welcome to <%= title %></p>
    <form action="/" method="post">
      <label for="name">Name:</label>
      <input name="name" type=input" />
      <button type="submit">Submit</button>
    </form>

    <% if (message) { %>
      <h3><%= message %></h3>
    <% } %>
  </body>
</html>

Note the EJS alligator tags (<% %> and <%= %>).

When all expectations are satisfied, you will see something like this:

1
2
3
4
5
6
7
8
Randomized with seed 17093
Started
.....


5 specs, 0 failures
Finished in 0.106 seconds
Randomized with seed 17093 (jasmine --random=true --seed=17093)

What’s next?

Got a test question? What kind of app should I build and test?

If you learned something, support my work through Wycliffe or Patreon.


A better open-source extension for Silhouette Cameo, Inkscape, and Ubuntu

This post demonstrates how to configure the open-source inkscape-silhouette extension on Ubuntu 18.04.

My previous method is documented here.

Even now this post is out of date, as Ubuntu 20.04 has almost certainly been released at the time of writing (though I am far too lazy to go check). It may also be out of date because I don’t see a lot of Silhoutte merchandise at the local craft store anymore. Is there a similar way to interface with the Cricut?

System and dependencies

Do the usual system prep before adding the software upon which Inkscape and the Silhouette extension depend:

1
2
sudo apt update
sudo apt upgrade

Ubuntu 18.04

Just as with a conventional printer, the Silhouette Cameo requires some drivers be installed before it can work with Ubuntu.

Open your System Settings:

Select Printers click Add:

Select your device and press Add:

You may see this:

Generic text-only driver automatically installed:

Inkscape

The Inkscape vector graphics tool has an extension that enables you to send your own SVG files to the Cameo.

Add the Inkscape repository and install:

1
2
3
sudo add-apt-repository ppa:inkscape.dev/stable
sudo apt update
sudo apt install inkscape

Run it from the command line to make sure it works:

1
inkscape

inkscape-silhouette extension

These steps are adapted from the inkscape-silhouette wiki.

This extension depends upon python-usb:

1
sudo apt install python-usb

Next, you’ll need to download a copy of the extension’s latest release. At the time of writing, you could obtain it from the command line like this:

1
2
3
cd ~
wget https://github.com/fablabnbg/inkscape-silhouette/releases/download/v1.22/inkscape-silhouette_1.22-1_all.deb
sudo dpkg -i inkscape-silhouette_1.22-1_all.deb

Try it out

Execute inkscape (from the command line, if you wish):

1
inkscape

Load the SVG file you want to cut and navigate to Extensions > Export > Send to Silhouette:

I leave the settings for you to play with. I only cut vinyl, so I go with the extension-provided defaults:

When ready, press Apply and watch your Silhouette Cameo spring to life.


A non-intrusive behavioural testing approach to bootstrapped React in Typescript

I’m on a team that loves Typescript and React. Their code is manually tested. I prefer to write my tests first.

The following process addresses the problem of respecting an existing approach to app development, while respecting my own standard of professional practice.

This is how I introduced my behavioural testing approach to a project bootstrapped with create-react-app. It equips me to write automated tests while allowing my teamates their own approach.

Generate a React Typescript app

As per the docs, the following produces the base application: 

1
2
npx create-react-app my-app --typescript
cd my-app

create-react-app comes with a few baked-in npm scripts. There are two existing build scripts: npm run build and npm start. The first only produces production builds. The latter only produces transient development builds (i.e., you can’t save them for later).

I need to produce test builds, so I add this to this to the scripts section in package.json:

1
2
3
4
5
6
"scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "build:test": "npx env-path -p .env.test react-scripts build"
...
}

create-react-app does a lot of crazy-cool stuff with .env files. I don’t think what follows will respect the dotenv-flow and dotenv-expand features that allows you to cascade build configurations according to environment.

With that it mind, install env-path:

1
npm install --save-dev env-path

I want a test build, so I need a .env.test file. Configurations, of course, are app dependent. The following is one likely example:

1
2
REACT_APP_REDIRECT_URI=http://localhost:3000
REACT_APP_CLIENT_ID=SomeExampleToken123

To create your test build, execute:

1
npm run build:test

You’ll see a message that says:

1
Creating an optimized production build...

Uh oh! react-scripts build only produces a production build. Is it still creating a production build, or is it reading my .env.test file?

This is the perfect opportunity to test if this test build configuration is working…

jasmine and zombie.js

None of this requires testing with jasmine and zombie, but I like testing with this pair because I’m old-fashioned and lazy. Install and initialize:

1
2
npm install --save-dev jasmine zombie 
npx jasmine init

NOTE: I’m purposefully skipping adding typescript support to jasmine for the moment. I may revisit this in the future.

Testing the build

I need to determine if the values I set in .env.test are being baked into the test build configured above.

Here’s my first test, which I place in spec/indexSpec.js:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
const path = require('path');
require('dotenv').config({ path: path.resolve(process.cwd(), '.env.test') });
require('../support/server');

const Browser = require('zombie');
const PORT = 3000;

Browser.localhost('example.com', PORT);

describe('landing page', () => {

  let browser, document;
  beforeEach(done => {
    browser = new Browser({ waitDuration: '30s', loadCss: false });

    // Wait for React to execute and render
    browser.on('loading', (doc) => {
      document = doc;
      document.addEventListener("DOMContentLoaded", (event) => {
        done();
      });
    });

    browser.visit('/', (err) => {
      if (err) return done.fail(err);
    });
  });

  it('displays the .env.test config variables', () => {
    browser.assert.link('a', 'Redirect', 'http://localhost:3000/&id=SomeExampleToken123');
  });
});

If you look closely at the above file, you’ll see this test needs its own server to run (npm start only does a development build). Paste the following into spec/support/server.js:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
const express = require('express');
const path = require('path');
const app = express();
const logger = require('morgan');

app.use(logger('dev'));

app.use(express.static(path.join(__dirname, '../../build')));

app.get('/*', function(req, res) {
  res.sendFile(path.join(__dirname, '../../build', 'index.html'));
});

const port = 3000;
app.listen(port, '0.0.0.0', function() {
  console.log(`auth-account listening on port ${port}!`);
});

You need morgan to see the server output:

1
npm install --save-dev morgan

Configure jasmine script

Add another line in the package.json (i.e., the e2e line right below the build:test line already configured):

1
2
3
4
5
6
7
"scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "build:test": "npx env-path -p .env.test react-scripts build",
    "e2e": "npm run build:test && npx jasmine"
...
}

The tests should now execute with one failing test:

1
npm run e2e

Make the test pass

The test defined above is simply checking to see if a link is created from the values stored in the .env.test file. Add that link to src/App.tsx:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
// ...

const App: React.FC = () => {
  return (
    <div className="App">
      <header className="App-header">
        <img src={logo} className="App-logo" alt="logo" />
        <p>
          Edit <code>src/App.tsx</code> and save to reload.
        </p>
        <a
          className="App-link"
          href="https://reactjs.org"
          target="_blank"
          rel="noopener noreferrer"
        >
          Learn React
        </a>
      </header>

      <!-- Add this link! Right here!!! -->
      <a href={`${process.env.REACT_APP_REDIRECT_URI}/&id=${process.env.REACT_APP_CLIENT_ID}`}>Redirect</a>
    </div>
  );
}

Execute the test:

1
npm run e2e

It passes!

More work…

This configuration doesn’t currently respect the neat dotenv-flow and dotenv-expand features that come baked into create-react-app.

As noted above, jasmine is not currently configured to support typescript.


Basic Android-React Native environment setup in Ubuntu 18.04

I am a test-driven developer who avoids fancy IDEs. I attempted to work through the details of a headless Android-React Native development environment, but quickly realized I was in over my head. This document outlines what may be the more typical workspace arrangement. It also demonstrates how I got everything working with Detox.

The following steps were executed on an Ubuntu 18.04 Desktop machine. What follows is heavily adapted from the Facebook and Detox.

Dependencies

Node

You need node 8.3 or newer. I’m using 10.15.3.

React Native CLI

1
npm install -g react-native-cli

Java JDK

This is the version recommended by Facebook. Installation instructions are adapted from those provided by DigitalOcean.

1
sudo apt install openjdk-8-jdk

Android Studio

You can download the IDE here. I simply installed via the Ubuntu Software manager.

On first execution, select Do not import settings and press _OK_. There are some Setup Wizard screens, which you can navigate. When given the opportunity, choose a Custom setup when prompted to select an installation type. Check the following boxes:

  • Android SDK
  • Android SDK Platform
  • Android Virtual Device

Click Next to install all of these components.

Configure SDK

A React Native app requires the Android 9 (Pie) SDK. Install it throught the SDK Manager in Android Studio. Expand the Pie selection by clicking the Show Package Details box. Make sure the follow options are checked:

  • Android SDK Platform 28
  • Intel x86 Atom_64 System Image _or_ Google APIs Intel x86 Atom System Image (I chose the first option)

Add the following lines to your $HOME/.bashrc config file:

1
2
3
4
5
export ANDROID_HOME=$HOME/Android/Sdk
export PATH=$PATH:$ANDROID_HOME/emulator
export PATH=$PATH:$ANDROID_HOME/tools
export PATH=$PATH:$ANDROID_HOME/tools/bin
export PATH=$PATH:$ANDROID_HOME/platform-tools

Load the config into the current shell:

1
source $HOME/.bashrc

Compile Watchman

1
2
3
4
5
6
7
8
sudo apt install libssl-dev autoconf automake libtool pkg-config python-dev
git clone https://github.com/facebook/watchman.git
cd watchman
git checkout v4.9.0  # the latest stable release
./autogen.sh
./configure
make
sudo make install

Install KVM

Adapted from here.

Check if your CPU supports hardware virtualization, by typing:

1
egrep -c '(vmx|svm)' /proc/cpuinfo

Install dependencies:

1
sudo apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils

Add your user to some groups, replacing by your own username:

1
2
sudo adduser $USER kvm
sudo - $USER

Check if everything is ok:

1
sudo virsh -c qemu:///system list

React Native CLI

1
npm install -g react-native-cli

Create a React Native project

1
react-native init AwesomeProject

Use Android Studio to open ./AwesomeProject/android. Open AVD Manager to see a list of Android Virtual Devices (AVDs).

Click Create Virtual Device, pick a phone (I picked Nexus 5), press Next, and select the Pie API Level 28 image (I had to download it first).

I run the emulator apart from the Android Studio environment:

1
~/Android/Sdk/emulator/emulator -avd Nexus_5_API_28

Execute the AwesomeProject app:

1
2
cd AwesomeProject
react-native run-android

Add Detox to Android project

Here, I simply consolidated all the setup steps described over several pages of Detox docs.

1
2
npm install -g detox-cli
npm install --save-dev detox

Paste to package.json:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
"detox" : {
  "configurations": {
    "android.emu.debug": {
      "binaryPath": "android/app/build/outputs/apk/debug/app-debug.apk",
      "build": "cd android && ./gradlew assembleDebug assembleAndroidTest -DtestBuildType=debug && cd ..",
      "type": "android.emulator",
      "name": "Nexus_5_API_28"
    },
    "android.emu.release": {
      "binaryPath": "android/app/build/outputs/apk/release/app-release.apk",
      "build": "cd android && ./gradlew assembleRelease assembleAndroidTest -DtestBuildType=release && cd ..",
      "type": "android.emulator",
      "name": "Nexus_5_API_28"
    }
  }
}

Configure Gradle

In android/build.gradle you need to add this under allprojects > repositories. The default init will look much like this already. Note the two separate maven blocks:

1
2
3
4
5
6
7
8
9
10
11
12
13
allprojects {
  repositories {
    // ...
    google()
    maven {
      // All of Detox' artifacts are provided via the npm module
      url "$rootDir/../node_modules/detox/Detox-android"
    }
    maven {
      url "$rootDir/../node_modules/react-native/android"
    }
  }
}

Set minSdkVersion in android/build.gradle:

1
2
3
4
5
buildscript {
    ext {
        // ...
        minSdkVersion = 18
        // ...

Add to dependencies in android/app/build.gradle:

1
2
3
4
5
dependencies {
  // ...
  androidTestImplementation('com.wix:detox:+') { transitive = true } 
  androidTestImplementation 'junit:junit:4.12'
}

Also in android/app/build.gradle, update defaultConfig:

1
2
3
4
5
6
7
8
9
android {
  // ...
  
  defaultConfig {
    // ...
    testBuildType System.getProperty('testBuildType', 'debug')  // This will later be used to control the test apk build type
    testInstrumentationRunner 'androidx.test.runner.AndroidJUnitRunner'
  }
}

Add Kotlin

In android/build.gradle, update `dependencies:

1
2
3
4
5
6
7
8
9
10
11
12
13
buildscript {
    // ...

    ext: {
      // ...
      kotlinVersion = '1.3.10' // Your app's version
    }
    
    dependencies: {
        // ...
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlinVersion"
    }
}

Create Android Test Class

Execute:

1
2
3
mkdir -p android/app/src/androidTest/java/com/awesomeproject/
wget https://raw.githubusercontent.com/wix/Detox/master/examples/demo-react-native/android/app/src/androidTest/java/com/example/DetoxTest.java
mv DetoxTest.java android/app/src/androidTest/java/com/awesomeproject/

At the top of the DetoxTest.java file, change com.example to com.awesomeproject.

Add testing frameworks

1
npm install mocha --save-dev

Create template example tests:

1
detox init -r mocha

Build app

1
detox build --configuration android.emu.debug

Run tests

Make sure the emulator is running:

1
~/Android/Sdk/emulator/emulator -avd Nexus_5_API_28

Start the react-native server:

1
react-native start

Run tests:

1
detox test -c android.emu.debug

Notes

Switch between projects I had difficulty with watchman. The instructions found here cleared the error:

1
2
watchman watch-del-all
watchman shutdown-server

Peace



Dockerized Matomo on Ubuntu 16.04

I’ve been hard on CloudAtCost before… they’re still terrible, but I’ve got a lot of use of my one-time purchase. I still use the resources I own to run non-critical applications. Matomo falls into that category.

Anyhoo, my server crashed and had to be deleted. This is how I setup Matomo on Ubuntu 16.04. Do it behind an nginx-proxy/lets-encrypt Docker Composition. This process is very manual and may one day be set up as a proper Docker build. As it stands, there is a lot of manual manipulation within the container.

First, create a project directory:

1
mkdir matomo && cd matomo

Copy and paste this into a file called docker-compose.yml.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
version: '3'
services:
  mariadb:
    image: 'bitnami/mariadb:latest'
    restart: unless-stopped
    environment:
      - ALLOW_EMPTY_PASSWORD=yes
      - MARIADB_USER=bn_matomo
      - MARIADB_DATABASE=bitnami_matomo
    volumes:
      - 'mariadb_data:/bitnami'
  matomo:
    image: 'bitnami/matomo:latest'
    restart: unless-stopped
    environment:
      - MATOMO_DATABASE_USER=bn_matomo
      - MATOMO_DATABASE_NAME=bitnami_matomo
      - ALLOW_EMPTY_PASSWORD=yes
      - MATOMO_USERNAME=Dan
      - MATOMO_EMAIL=someguy@example.com
      - VIRTUAL_HOST=matomo.example.com
      - LETSENCRYPT_HOST=matomo.example.com
      - LETSENCRYPT_EMAIL=someguy@example.com
    depends_on:
      - mariadb
    volumes:
      - 'matomo_data:/bitnami'
      - './misc:/opt/bitnami/matomo/misc/'
volumes:
  mariadb_data:
    driver: local
  matomo_data:
    driver: local
networks:
  default:
    external:
      name: nginx-proxy

Create and execute the container with:

1
docker-compose up -d

This is the time to start (or restart) the nginx-proxy/lets-encrypt composition. Once this is running, your username will be what was set in the docker-compose.yml file described above. In this case, the default credentials are:

  • Username: Dan
  • Password: bitnami

You should be able to login at the domain specified now.

App-level Configuration

matomo works out of the box, but there will be a bunch of things you’ll want to set up at the application level.

Upgrade

Before all that, there’s a weird permissions issue in the container. You’ll want to upgrade matomo, but won’t be able to do so until you fix this. It’s super hacky having to do this from within the container, but that’s what I’m working with at the moment.

From your project directory: 

1
docker-compose exec matomo bash

Then, from within the container:

1
2
chown -R daemon:daemon /opt/bitnami/matomo
chmod -R 0755 /opt/bitnami/matomo

Dependencies and Headers

Again, this is super hacky, because now you need to install an editor and a bunch of other dependencies within the container:

1
2
3
apt update
apt install vim git wget autoconf gettext libtool build-essential
vim /opt/bitnami/matomo/config/config.ini.php

Add this to the [General] section:

1
2
3
4
5
force_ssl = 1

; Standard proxy
proxy_client_headers[] = HTTP_X_FORWARDED_FOR
proxy_host_headers[] = HTTP_X_FORWARDED_HOST

Exit the container and restart.

1
docker-compose restart

Config checklist

At this point, everything should be operational on a basic level. Address the following points, and get a lot more use out of matomo.

Personal > Settings

  • Change password
  • Exclude your own visits using a cookie

System > Geolocation

I set up the GeoIP2 (Php) extension, which is supposed to make things faster somehow.. 

1
docker-compose exec matomo bash

From within the container, clone libmaxminddb:

1
git clone --recursive https://github.com/maxmind/libmaxminddb

Install from inside the cloned directory:

1
2
3
4
5
6
cd libmaxminddb
./bootstrap 
./configure
make
make install
ldconfig

Install the extension:

1
2
3
4
5
6
7
cd ..
git clone https://github.com/maxmind/MaxMind-DB-Reader-php.git
cd MaxMind-DB-Reader-php/ext
phpize
./configure
make
make install

Edit php.ini:

1
vim /opt/bitnami/php/lib/php.ini

Add this to the end and save:

1
extension=maxminddb.so

Get the database:

1
2
3
4
cd /opt/bitnami/matomo/misc
wget https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz
tar -xvfz GeoLite2-City.tar.gz
mv GeoLite2-City_20190115/* .

Exit the container and restart from the host:

1
docker-compose restart

If you refresh the System > Geolocation page, GeoIp 2 (Php) will be operational. Select this option and save.

Websites > Manage

Add all the websites you want track.

Conclusion

I needed to bang this out for my own purposes. I will likely be forced to revisit this when CloudAtCost fails me once again.