Author: ecadmin

It is often useful to start up a Docker container running a database server with a pre-defined, ready and prepopulated database via an SQL script, usable as soon as the container starts.

This can be a dependency for local development, a dependency for tests, among others.

For macOS: if it is not present, install MySQL with Homebrew to get the MySQL client.

brew install mysql

To define the container we need two files in the same directory. The Dockerfile extending mysql and specifying the start script:

Dockerfile:

FROM mysql

COPY ./create-local-db.sql /tmp

CMD [ "mysqld", "--init-file=/tmp/create-local-db.sql" ]

The SQL script to define the database in the container:

create-local-db.sql:

-- Local database definition.

DROP DATABASE IF EXISTS local_db;

CREATE DATABASE local_db;

USE local_db;

DROP TABLE IF EXISTS books;

CREATE TABLE books (
  id int(10) NOT NULL,
  title varchar(30) NOT NULL DEFAULT '',
  PRIMARY KEY (id)
);

INSERT INTO books VALUES(1, 'Book 1');
INSERT INTO books VALUES(2, 'Book 2');

Build the container in the directory with Dockerfile, tagging it with the name my_db (for example):

docker build -t my_db .

Run the container on port 3306:

docker run -e MYSQL_ROOT_PASSWORD=pw -p 3306:3306 my_db

Note that we must pass the root password environment variable to the server.
(For docker-compose this would go under environment: )

The server should indicate that it is ready for connections.

In another terminal, connect to the server with MySQL client:

mysql --host=127.0.0.1 --port=3306 -u root -p

You should see the MySQL prompt and be able to run queries.

mysql> use local_db;
Database changed

mysql> show tables;
+--------------------+
| Tables_in_local_db |
+--------------------+
| books |
+--------------------+
1 row in set (0.00 sec)

The MySQL database inside the Docker container is ready to use.

We can output a visualization of a Keras deep learning model using the plot_model utility function.

First the dependencies need to be installed:

$ pip install pydot
$ pip install pydotplus
$ pip install graphviz

We also need the graphviz binary:

$ sudo port install graphviz

It is also possible to use Brew (brew install graphviz).

This example code outputs a model diagram:

from keras import layers
from keras import models
from keras.utils import plot_model

network = models.Sequential()

network.add(layers.Dense(512, activation='relu', input_shape=(28*28,)))
network.add(layers.Dense(10, activation='softmax'))

plot_model(network)

This will generate the model plot in a PNG file named model.png by default.

View the result diagram image:

$ open model.png

Result image:

 

We can also specify a different file name in the call:

plot_model(network, to_file='my-model.png')

To also include layer shapes in the plot, omitted by default:

plot_model(network, show_shapes=True)

The following code shows how to read a video frame from the webcam.

This assumes the OpenCV library for computer vision is installed. See the post on Installing OpenCV on macOS if needed.

To capture one image:

import cv2
import matplotlib.pyplot as pyplot

captureObject = cv2.VideoCapture(0)

success, testImage = captureObject.read()

# See image numeric values.
print(testImage)

# Plot image.
pyplot.imshow(testImage, cmap='gray')
pyplot.show()

To capture many images continuously, we just need to call read() in a loop.

 

To install the OpenCV library for computer vision on macOS, first install the required prerequisites, then the package itself.

pip install scikit-build

pip install cmake

pip install opencv-python

Test the installation:

python

>> import cv2

The import should work without errors.

 

To export data from a table T (or any result of a query) in a PostgreSQL database use the following command:

$ psql -h localhost
       -U database_usr
       -d database_name
       -c "COPY (SELECT * FROM T) 
           TO '/path/file.csv' 
           DELIMITER ',' CSV HEADER;"

Notes:

• The query should be on one line; it is split here for readability.
• Make sure you have the correct permissions to write to the directory referenced. The command may need to be run with sudo.

 

Testing code which calls APIs often involves handling various HTTP status codes from an endpoint.
When calling an external service, it may be difficult to trigger specific HTTP status codes for errors to test out code paths triggered only by those status codes being returned.
We can call the service httpstat.us with the desired response code in the URL path to get any arbitrary HTTP status code returned.

For example, using cURL:

$ curl -v http://httpstat.us/503

The result is:

> GET /503 HTTP/1.1
> Host: httpstat.us
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable
...

Less common success status codes can be simulated as well:

$ curl -v http://httpstat.us/204

Result:

> GET /204 HTTP/1.1
> Host: httpstat.us
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 204 No Content
...

In a config file, we can use httpstat.us URLs instead of our actual dependencies to trigger error cases for tests.

See the full feature list of the HTTP Status service:

http://httpstat.us/

Transitive Inclusion (Transclusion for short) is a feature of REST APIs useful for clients wishing to get more than just the resource in the URL path, in one request.
We use a query string parameter called inline, or perhaps include, equal to the name of the associated objects to include.

For example, suppose that in a Books API, author objects have associated books.
A request for an author:

GET /authors/12345

{
  "id": 12345,
  "name": "Ralph Ellison"
}

The list of books associated with the author:

GET /authors/12345/books

[
  {
    "id": 1,
    "title": "Invisible Man"
  },
  {
    "id": 2,
    "title": "Juneteenth"
  }
]

We can pass the inline parameter on an author request:

GET /authors/12345?inline=books

{
  "id": 12345,
  "name": "Ralph Ellison",
  "books": [
    {
      "id": 1,
      "title": "Invisible Man"
    },
    {
      "id": 2,
      "title": "Juneteenth"
    }
  ]
}

Note the extra field “books” is included or inlined in the author object.
This way a client app can get the author object and associated books in one request, reducing round-trips. Of course if the objects are small, including the associated objects by default is the best option; this is mostly useful for large, expensive-to-retrieve associated objects.

The transitive inclusion feature can be even more powerful if we get a list of all authors in the API, inlining all of their associated books:

GET /authors?inline=books

Or we can include many associated objects:

GET /authors?include=books,articles

The result is a comprehensive list object, all in one request:

[
  {
    "id": 12345,
    "name": "Ralph Ellison",
    "books": [
      {
        "id": 1,
        "title": "Invisible Man"
      },
      {
        "id": 2,
        "title": "Juneteenth"
      }
    ],
    "articles": [ ... ]
  },
  {
    "id": 67890,
    "name": "Doris Lessing",
    "books": [
      {
        "id": 10,
        "title": "The Golden Notebook"
      },
      {
        "id": 11,
        "title": "The Fifth Child"
      },
      ...
    ],
    "articles": [ ... ]
  },
  ...
]

Such a comprehensive response object can be convenient for an app to build a page listing authors and books, for example; only a single HTTP request can be used .
Note that if the set is large, pagination can be used in addition.

 

Using Node on the server side, we find that we cannot use the JavaScript functions atob and btoa. They are not defined; we get the following errors:

ReferenceError: btoa is not defined

ReferenceError: atob is not defined

The functions are defined in JavaScript in the browser, but not in server-side Node.js.

To encode to base-64, instead of btoa use the following:

const myString = 'hello'
const encoded = Buffer.from(myString)
                      .toString('base64')
console.log(encoded)

Output:

aGVsbG8=

To decode from base-64, instead of atob use the following:

const base64String = 'aGVsbG8='
const decoded = Buffer.from(base64String, 'base64')
                      .toString('utf-8')
console.log(decoded)

Output:

hello

 

Recall that using jest.mock(module) mocks every function exported from the module.

But sometimes we need to mock only one or some functions in a module, leaving the others’ real implementations as-is.

Suppose we are using the following module of functions:

my-functions.js:

const f = () => 'Return from f'
const g = () => 'Return from g'
const h = () => 'Return from h'

module.exports = {
  f,
  g,
  h
}

The code below uses a Jest spy with a mock implementation to mock out just one function while leaving alone the original implementations of the other two.

my-functions.test.js:

const functions = require('./my-functions')

describe('mock only one function from module', () => {
  it('should return only one mocked result', () => {
    jest.spyOn(functions, 'g')
      .mockImplementation(() => '_mock_')

    console.log(functions.f())
    console.log(functions.g())
    console.log(functions.h())

    expect(functions.f()).toEqual('Return from f')
    expect(functions.g()).toEqual('_mock_')
    expect(functions.h()).toEqual('Return from h')
  })
})

Running the tests:

$ jest my-functions.test.js
PASS ./my-functions.test.js
mock only one function from module
✓ should return only one mocked result (14ms)

console.log my-functions.test.js:8
Return from f

console.log my-functions.test.js:9
_mock_

console.log my-functions.test.js:10
Return from h

Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: 0.956s
Ran all test suites matching /my-functions.test.js/i.

Note that only g() returns a mocked string.

 

A Jest mock function can be set to return a specific value for all calls, or just once.

Note that if we define a return value with mockReturnValueOnce, the mock function will return undefined for all subsequent calls.
It will not throw an error and may go unnoticed, causing undesirable behaviour.

If a Jest mock returns undefined incorrectly check to make sure how it was defined and how many times it was called.

The code below shows the different behaviours:

describe('mock once tests', () => {
  test('many returns', () => {
    const f = jest.fn()
    f.mockReturnValueOnce('result')

    const result1 = f()
    console.log(result1)

    const result2 = f()
    console.log(result2)

    const result3 = f()
    console.log(result3)
  })

  test('one return', () => {
    const f = jest.fn()
    f.mockReturnValue('result')

    const result1 = f()
    console.log(result1)

    const result2 = f()
    console.log(result2)

    const result3 = f()
    console.log(result3)
  })
})

Output:

result
undefined
undefined

result
result
result