Node.js feature-flipping through Git

Git as a Continuous Manager

@m4d_z
alwaysdata logo
I have a codebase
Which is used by a lot of people
And I pushed a feature in the auth system
And we locked out a lot of users :(

My last time with that was a month ago with GitHub

We deliver in a continuous way, so need a quick rollback system

What are…?

Feature Flags

a.k.a. Feature Flipping
Feature Flags
Feature toggle is used to hide, enable or disable the feature during run time.
if ( ff.flag('crazy') ) {
  doCrazyExperiment()
} else {
  doReallySafeStuff()
}

Feature Flag vs. Branching

ACL

One list to rule them all
// guest is allowed to view blogs
acl.allow('guest', 'blogs', 'view')
// allow function accepts arrays as any parameter
acl.allow('member', 'blogs', ['edit', 'view', 'delete'])
// test jsmith permissions
acl.isAllowed('jsmith', 'blogs', ['edit', 'view', 'delete'])
// check james permissions
acl.allowedPermissions('james', ['blogs', 'forums'], (err, perms) => {
  console.log(perms)
})
// run ACL as a middleware
app.put('/blogs/:id', acl.middleware(), (req, res, next) => { /*...*/ })

Database Migrations

Upgrade to database, now!
Database Migration
A schema migration is performed on a database whenever it is necessary to update or revert that database’s schema to some newer or older version.
module.exports = {
  up () {
    return new Promise((resolve, reject) => { /*...*/ })
  },
  down () {
    return new Promise((resolve, reject) => { /*...*/ })
  }
}
$ npm run db:migrate [up|down] {version}

(Git) Hooks

Not this hook...

Functional (Reactive) Programming

_('click', $('#cats-btn'))
  .throttle(500)	// can be fired once in every 500ms
  .pipe(getDataFromServer)
  .map(formatCats)
  .pipe(UI.render);

Git --bare

Init

Server side

$ ssh admin@staging
|admin@staging| $ git init --bare /srv/git/my-project.git
|admin@staging| $ tree -L 2 /srv/git/my-project.git
/srv/git/my-project.git
|-- branches
|-- hooks
|   |-- post-checkout
|   |-- post-commit
|   |-- post-merge
|   |-- post-receive
|   |-- post-receive.d
|   |-- pre-push
|   |-- update
|   `-- update.d
|-- index
|-- info
`-- refs
    |-- heads
    `-- tags

On developers side

[~my-project] $ git add remote staging git@staging:/srv/git/my-project.git

Hooks scripts

#!/bin/sh
# $GIT_BARE_REPOSITORY/hooks/post-receive
GIT_DIR=$(dirname $PWD)
while read oldrev newrev ref
do
	# Load tasks from hooks/post-receive.d/*
	if test -d "$PWD/post-receive.d"
	then
		for task in "$PWD/post-receive.d/*.sh"
		do
			test -r $task && source $task
		done
		unset task
	fi
done

Filter

#!/bin/sh
# $GIT_BARE_REPOSITORY/hooks/update

while read oldrev newrev ref
do
	if [[ $ref = refs/heads/production ]]
	then
		# Commands to run on `production` branch only
	fi
done

Deploy

#!/bin/sh
# $GIT_BARE_REPOSITORY/hooks/post-receive.d/10-deploy

$RUN_DIR="/srv/my-app"

if [[ $ref = refs/heads/production ]]
then
	git --work-tree=$RUN_DIR \
		--git-dir=$GIT_DIR \
		checkout --force production
fi

npm scripts

#!/bin/sh
# $GIT_BARE_REPOSITORY/hooks/post-receive.d/20-npm-run-migrate

$RUN_DIR="/srv/my-app"

if [[ $ref = refs/heads/production ]]
then
	cd $RUN_DIR && npm run db:migrate up 2018-nodejs-it
fi

Git Push!

[~/my-project] $ git push staging production

How to inform the workers about the changes?

Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features”.

Doug McIlroy, in Bell System Technical Journal, 1978, 1th rule

Signals!

Unix-socket:

echo "SIGNAL db:migrated" | /usr/bin/socat - UNIX-CONNECT:/var/run/my-app.sock

Use a Node.js instance as coordinator

const net = require('net')
const socket = net.createServer(sock => {
    sock.write(`Hello ${sock.remoteAddress}\n`)
    // when `socat` writes a message
    sock.on('data', data => {
        if (data === 'SIGNAL db:migrated') {
          //...
        }
    })
})
socket.listen('/var/run/my-app.sock')

Use streams for FRP

What if…?

  • Node.js coordinator instance crashes? → Restart it!
  • Need to scale? → Detached workers
  • We need to rollback database → Never be destructive on your database
  • I just need to update ACL / Flags → Filter your Git commits

How to broadcast/dispatch the actions?

Broker!

  • ØMQ
  • Redis
  • MQTT
  • etc.

Self-reload Workers

const {fork} = require('child_process')

(function main () {
  const server = fork('server.js').unref()

  sock.connect('tcp://127.0.0.1:3000')
  sock.on('RESTART', () => {
    server.kill()
    setTimeout(main, 1000)
  })
})()

1/ Init ACL/Flags

const flags = ff.config({
  criteria () { return [
    { id: 'userInGroup',
      check (user, group) { acl.isAllowed(user.id, group, '*') }},
    { id: 'environment',
      check (user, env) { return process.env.NODE_ENV === env }}
  ]},
  features () { return [
    { id: 'admin',
      criteria: { userInGroup: 'admin' }},
    { id: 'development',
      criteria: { environment: 'development' }}
  ]}
})

2/ Reload ACL/Flags

sock.connect('tcp://127.0.0.1:3000')
sock.on('ACL:RELOAD', ({perms, features}) => {
  controls.removeAllow('*', '*', '*')
  controls.allow(perms)
  flags.config({features})
  flags.reload()
})

Enable API endpoints on-the-fly

// hook to initialize the endpoint route at runtime
app.post('/api/:endpoint', (req, res) => {
  if (req.params.endpoint === 'list') {
      if (flags.isFeatureEnabledForUser('list', user)) {
        let listController = require('controllers/ListController')
        listController.init(app)
        res.status(200).send()
      } else {
        res.status(401).send()
      }
  }
})
app.get('/api/:endpoint', (req, res) => { /*...*/})

Use HTTP Codes correctly

  • 204: No Content
  • 303: See Other
  • 307: Temporary Redirect
  • 401: Unauthorized
  • 403: Forbidden

Feature Flags & ACL

Server side

Inside the workers

1/ REST API

  • Use dynamic endpoints
  • Serve relevant ACL through API
  • Send flags via API
  • Use signed payloads

2/ Update JWT

  • Send updated JWT to your Client
  • Ask your clients to re-auth
  • Upgrade permissions/flags in JWT payload

3/ Block/Redirect requests

  • Properly deny resources access
  • Handle the use-case in your Client
  • Self-reload client in background

Client side

In your PWA

WebSocket

Push API !

const subscription
const payload
const options = {
  TTL: 3600
}

webPush.sendNotification(subscription, payload, options)
.then(() => res.sendStatus(201))
.catch(err => res.sendStatus(500))

Service Worker/Notification

self.addEventListener('push', event => {
  const payload = event.data ? event.data.text() : 'no payload'
  event.waitUntil(

    // Reload ACL/flags

    self.registration.showNotification('ACL Updated!', {
      body: payload
    })
  )
})

Best practices & more

Code Splitting

const getComponent = () => {
  return flags.isFeatureEnabledForUser('component', user)
    ? import(/* webpackChunkName: "lodash" */ 'lodash')
      .then(({ default: _ }) => {
        let element = document.createElement('div')
        element.innerHTML = _.join(['Hello', 'world'], ' ')
        return element
      })
      .catch(error => 'An error occurred while loading the component')
    : Promise.reject(new Error(403))
}

getComponent().then(component => { /*...*/ })

Simple Monitoring View

Unleash Server Monitoring View
Unleash Server Monitoring View

DevOps are here to ensure your users to not suffer a mean time in your continuous delivery. And Node.js, with Streams and FRP, is a great tool for DevOps tasks.

m4dz's avatar
m4dz

Paranoïd Web Dino · Tech Evangelist

alwaysdata logo
https://www.alwaysdata.com

Questions?

Thank You!


Available under licence CC BY-SA 4.0

Illustrations

m4dz, CC BY-SA 4.0

Interleaf images

Courtesy of Unsplash and Pexels contributors

Icons

  • Layout icons are from Entypo+
  • Content icons are from FontAwesome

Fonts

  • Cover Title: Sinzano
  • Titles: Argentoratum
  • Body: Mohave
  • Code: Fira Code

Tools

Powered by Reveal.js

Source code available at
https://git.madslab.net/talks