Flexible and reusable functions

Yoann Gendrey -

I am working since few years in Node.js and since I am working at Pops, I had to work with TypeScript. TypeScript is great like compiler that will translate all of your *.ts TypeScript source code will be compiled into plain Javascript. There are many advantages to use TS the major ones according to me are the following ones:

  • Your development will be supercharged with types (number, boolean, string etc...)

eg:Normal JavaScript vs TypeScript

const counter = 0
const counter: number = 0
  • You will be able to use ECMAScript 5/6/7

The first one will guarantee consistency in your development and type checking. The other one will let you use the new 'era' best practices of Javascript such as using arrow functions.

const sum: number = (a: number,b: number) => a+b

More info here:
https://en.wikipedia.org/wiki/TypeScript
https://www.typescriptlang.org

If you are a developer you probably heard about DRY (Do not Repeat Yourself). I am writing some functions and I figured it out a nice pattern.

Later I will speak about Async:
http://caolan.github.io
https://www.npmjs.com/package/async

I'm using this module all the time and you probably should too.

The idea behind this 'pattern' is the following one. Often you will have to apply a function on a set of elements. By definition it works like this:

INPUT -----FUNCTION-----> OUTPUT

The pattern is saying rather than designing a FUNCTION for a specific INPUT wich is unique, you should or even have to design it for a bunch of INPUT meaning.

N INPUT -----FUNCTION-----> N OUTPUTS

How to do this in Node.js

Simply deals with arrays of inputs. They are called iteratees, then get the result and deal with this all along your process. It means you can deal with n items where n ∈ [0; +∞[. This is pretty interesting because when your function is designed that way and ready to handle either 0, 1 or n bunch of items it should work almost all the time without taking care of size of the input. Also when you work with Async most of the time it has an iteratee as input. Async actually helps to deals and controls your flow when you are dealing with operations that takes time. Basically you'll get:

  • map/mapSeries
  • concat/concatSeries
  • Each/EachSeries
  • forEach/ForEachSeries
  • eachOf/eachOfSeries
  • every/everySeries
  • whilest

And many more, this is the ones I am using the most. The 'Series' ones are just the associated functions which are doing the job synchronously.

I wanted to illustrate what I was saying. For example I am looking for a function that will download and store resources locally. I called it getRessource and it is working like this.

Basically in the following example I am working on AWS with S3 service.
I also have a File factory function where I can dynamically get my s3 parameters from a pre defined File object.

const getRessource: any = (data: any) => {
 return new promise((resolve: any, reject: any) => {
  const get = (file: any, callback: any) => {
   const writeStream = fs.createWriteStream(file.path)
   const readStream = s3.getObject(file.s3Params().get).createReadStream()
   pump(readStream, writeStream, (err: any) => {
    if (err) {
     callback(err)
    } else {
     callback()
    }
   })
  }

  async.each(data.items, get, (err: any) => {
   if (err) {
    reject(err)
   } else {
    resolve(data)
   }
  })
 })
}

getRessource is:

  • returning a promise you will be able to get your resource and make an action later and also catch some errors
  • composed by two parts:
  • An action
  • A processor which is an Async function which can be either a map/each/every depending on what you are looking for after the process

You can also note that I am using pump. This by far one of the best to deals with streams. You don't have to take care about stream status when it close/fail or finish pump will handle it for you.

Behind this idea I am saying that obviously the easier your functions are the better it is. However when you can design flexible functions where it can handle n items you must write it down in this way. It will be easier to reuse those functions later.

Also from time to time you may have a single item if by error you are providing a non array item as items it will not work. You can do this also:

data.items = (Array.isArray(data.items)) ? data.item : [data.items]

This will turn your INPUT into an array with a single item. However I recommend you to take care of this and just by default provide the right input directly.

Later I may illustrate with a detailed example. I recently worked on a Serverless Zip AWS Lambda, and I have some tips if you need to work on that kind of stuff.

Thanks for reading.