Category Archives: Cheat sheet

Quick tutorials

Using Nginx as a reverse proxy and add source code replacement

Task:

I have web app (in my case Node app running on port 3000)
I would add Nginx in front of it and will serve it on port 80.

Adding reverse proxy config.

Find Nginx.config file by executing: nginx -V

Edit nginx.conf and edit the server section

...
    server {
        listen       8081;
        server_name  toni-develops.com;
        location / {
            proxy_pass http://localhost:3000/;
        }
...

Line 5 tells enginx to intercept all requests / and proxy them from http://localhost:3000
If the request from example is /home Nginx will make request to http://localhost:3000/home

If for example we want to proxy only specific urls (let’s say only *.php) we could do this:

location ~ \.php {
    proxy_pass http://localhost:3000/
}

Add source code replacement

Nginx has to be built with sub_filter module.
The example below will override all localhost:3000 to toni-develops.com:80

...

server {
    listen       8081;
    server_name  toni-develops.com;


    location / {
        proxy_pass http://localhost:3000/;
        proxy_set_header Accept-Encoding "";
        proxy_buffers 8 1024k;  
        proxy_buffer_size 1024k;
        sub_filter 'localhost:3000'  'toni-develops.com:80';
        sub_filter_once off;            
    }

...

 

Simple caching using browser’s service workers.

A service worker API is relatively new technology, allowing for running a special type of web worker (a JavaScript), that can be installed in the browser and could provide special features to the website like caching data and allowing for offline load.

Service workers are also heavily used in progressive apps since they provide functionalities for caching resources and push notifications.

Before we jump into building an example app, let’s see what are the main service worker properties.

Service worker properties:

– Service workers are plain JavaScript APIs.

– Service workers run on separate thread so they can’t access the elements of the main page like the DOM elements but this is not the intend of how they are used.

– Service workers can listen on an events emitted from the browser and execute an appropriate script. In example a service worker could listen on browser fetch event and fetch the data or return it from cache.

Service worker lifecycle.

On a first page visit the only thing that happens is that service worker got installed. No code inside the service worker got executed, unless the user hit another page from this website or reloads the current page. Then service worker kicks in and could provide the extra functionality to the site.

Let’s get started by creating a simple web page, that will make an AJAX request to this service: https://www.toni-develops.com/external-files/examples/service-workers/delayed-response.php that will return the current time, and they display it.

The service intentionally delays the response 5 seconds so this way we could observe when we open the page for the first time how the response is delayed, since service worker is installing and don’t have the response cached. Then if we reload the page the service worker will intercept the request and will serve last cached response right away and then  will make a request to the service and update the page.

Building a caching layer for the web resources using service worker.

Let’s create the page.

index.html

<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>title</title>
    <script
          src="https://code.jquery.com/jquery-3.3.1.min.js"
          integrity="sha256-FgpCb/KJQlLNfOu91ta32o/NMZxltwRo8QtmkMRdAu8="
          crossorigin="anonymous"></script>
    <link rel="stylesheet" href="style.css">
    <script src="script.js"></script>
  </head>
  <body>
    My fancy new site !
    <hr>
    <div id="container"></div>
  </body>
</html>

 

script.js

if ('serviceWorker' in navigator) {
  window.addEventListener('load', function() {
    navigator.serviceWorker.register('./sw.js').then(function(registration) {
      // Registration was successful
      console.log('ServiceWorker registration successful with scope: ', registration.scope);
    }, function(err) {
      // registration failed :(
      console.log('ServiceWorker registration failed: ', err);
    });
  });
}



$.ajax({url: "https://www.toni-develops.com/external-files/examples/service-workers/delayed-response.php", success: function(result){
  $("#container").html(result);
}});

 What’s happening here:
– line 1: we check if the browser supports service workers.
– line 2: added event listener, listening for load event and on load it installs the service worker. The rest is just error handling.
-line15:we make a request to the service using plain jQuery AJAX request.

style.css

body {
  background: silver
}

Now it’s time to write the actual service worker.

sw.js

var CACHE_NAME = 'my-site-cache-v1';
var urlsToCache = [
  './index.html',
  './style.css',
  './script.js',
  'https://www.toni-develops.com/external-files/examples/service-workers/delayed-response.php',
  'https://code.jquery.com/jquery-3.3.1.min.js'

];

self.addEventListener('install', event => {
  // Perform install steps
  event.waitUntil(
    caches.open(CACHE_NAME)
      .then(cache => {
        console.log('Opened cache');
        cache.addAll(urlsToCache);
      })
  );
});


self.addEventListener('fetch', event => {
  console.log("(9)served from service worker: ",event.request.url);
  // serve as soon as possible from cache
  event.respondWith(fromCache(event.request));
  // update cache
  event.waitUntil(  
    update(event.request)
  );
});


/**
 * 
 * Helper methods
 */

function fromCache(request) {
  return caches.open(CACHE_NAME).then(cache => {
    return cache.match(request);
  });
}


function update(request) {
  caches.open(CACHE_NAME).then( cache => {
    fetch(request).then( response => {
      cache.put(request, response)
    });
  });
}

What we just did:
– line 1: we specified the cache key (we have to make sure that it is unique)
– line 2: we created an array urlsToCache having all URLs that we want to cache. If one of the URLs fail, service worker will fail to install so we have to keep this url list short.
– line 11: added event listener, that on install event (when the service worker is invoked for the first time) will open the cache (line 14), and will add all urls to the cache (line 17) and then return the contents.
– line13: will simply prevent service worker to be killed before finishing with caching the resources.
– line 23: added fetch event. This will fires wherever a web page request a resource (*.js, *.css, images or in this case AJAX request)
– line 26: will return the asset right away from cache
– line29: will fetch the asset from the ‘slow’ service to update the cache for the next request.
Now let’s look at the helper methods:
fromCache function is simply opens the cache and finds the request key.
update function is opening the cache, fetches a fresh data from the service (or stores a fresh copy of web resource) and puts it back into the cache using request as a key.

Testing the project.

Open an empty tab in Chrome (comand + T), make sure that the dev tools are also open on the net tab (alt + comand + i), and the console drawer is visible (escape toggles the drawer)

Copy the example URL example found here and paste it into Chrome’s tab.

You will see the message “My fancy new site !” and after around 5 seconds current date and time will appear (- 5 seconds that took for the service to return the data)

Now service worker is installed. Reload the page and you will see that the current time will appear instantly from the cache. Observe the net tab and you will notice that the service worker also made a request to the service to update the data (the one that is pending on the picture below) and also web resources are now coming from the service worker (the size column).

Additionally if you want to clear the cache you could do it by clicking on the Application tab in the dev tools “Clear Storage” and then “clear site data”.

Simple use of Promises in JavaScript

Here is a simple and a bit silly example of using promises. In real life we could just move the logic into done and fail functions which are promises themselves but for the sake of explaining how promises work we are going to do this example.

you could see the example here.

 var promise = new Promise(function(resolveFunc, rejectFunc) {

    $.ajax({
      url: "https://www.toni-develops.com/external-files/examples/service-workers/delayed-response.php",
    })
      .done(function( data ) {
        resolveFunc(data);
      })
      .fail(function(error) {
          rejectFunc(error);
      });

});

promise.then(
    function(result) { // resolve func
        console.log('Success! Data: ', result);
}, function(error) {    // reject function
    console.log("Error! ", error);
})

console.log("Promise was called, and once it's fulfilled the code inside the 'then' function will be executed!");

What’s happening here:
– Line 1 creates a new promise, which does AJAX request and calls done or fail functions.
– If execution is successful and done function is called, it also invokes resolveFunc passing the result data.
– Line15 is where the promise has been called, and the bodies of resolveFunc (line 16) and  rejectFunc (line 18) are defined and passed to the promise.

Run the code and you should see first the message “Promise was called, and once it’s fulfilled the code inside the ‘then’ function will be executed!
Five seconds later the newly fetched content will be shown since the service intentionally takes 5 seconds to return the data for testing purposes.

What if we want to make it much easier? Let’s use async and await

async function fetchData() {
    var result = await fetch('https://www.toni-develops.com/external-files/examples/service-workers/delayed-response.php');
    return result;
}


(async () => {
    var r = await fetchData();
    console.log("ONE::", r);    
    console.log("DONE !");
})();

 

Listen for DOM changes using Mutation Observer

If we ever needed to listen for DOM events, one of the first options that comes to our mind might be to use setTimeout and periodically check the contents of the tag that we are interested in. Like in this example:

function myTest() {
    setTimeout(function() {
        console.log("Beep ...");
        if(document.getElementById('root').innerHTML == 'Loading ...') {
            myTest();
        }
        else {
            // do something 
            alert("READY!");
        }
    }, 1000);
}

But this is intense JavaScript process, which could be done using more elegant way with the help of Mutation Observer. 

Using Mutation Observer we could subscribe for different types of DOM events and trigger function on change.

example code:

<html>
  <body>    
    <div id="root">loading ...</div>

    <script>
      setTimeout(function() {
        document.getElementById('root').innerHTML = "My content is loaded!";
      }, 3000);
    </script>

    <script>
      var targetNode = document.getElementById('root');
      var config = { attributes: true, childList: true, subtree: true, characterData:true };
      var observer = new MutationObserver(function() {
        alert("#root div changed!");  
        // stop observing
        observer.disconnect();      
      });

      observer.observe(targetNode, config);
    </script>
  </body>
</html>

 what is happening here:
– created a div tag with id=’root’ and added ‘Loading …’ text.
– created MutationObserver (line 14) to listen for DOM changes in ‘root’ folder.
– specified which DOM events will trigger the function in config param (line 13)

– after 3 sec. setTimeout event fires and changes the content of the ‘root’ div, triggering DOM change event which will cause the function set in MutationObserver (line 14) to be executed.
– after showing the alert we could optionally disconnect and unsubscribe for DOM change events for this tag.

Here are some of the most used config properties

Mutation observer init properties:

At a minimum, one of childListattributes, and/or characterData must be true when you call observe(). Otherwise, a TypeError exception will be thrown.

attributeFilter Optional
An array of specific attribute names to be monitored. If this property isn’t included, changes to all attributes cause mutation notifications. No default value.
attributeOldValue Optional
Set to true to record the previous value of any attribute that changes when monitoring the node or nodes for attribute changes; see Monitoring attribute values in MutationObserver for details on watching for attribute changes and value recording. No default value.
attributes Optional
Set to true to watch for changes to the value of attributes on the node or nodes being monitored. The default value is false.
characterData Optional
Set to true to monitor the specified target node or subtree for changes to the character data contained within the node or nodes. No default value.
characterDataOldValue Optional
Set to true to record the previous value of a node’s text whenever the text changes on nodes being monitored. For details and an example, see Monitoring text content changes in MutationObserver. No default value.
childList Optional
Set to true to monitor the target node (and, if subtree is true, its descendants) for the addition or removal of new child nodes. The default is false.
subtree Optional
Set to true to extend monitoring to the entire subtree of nodes rooted at target. All of the other MutationObserverInit properties are then extended to all of the nodes in the subtree instead of applying solely to the target node. The default value is false.

Passwordless SSH log in for MAC OS X

On the machine that you want to use to ssh to the other OS X machine do this:

– Create id_rsa key.
when prompted use the suggested filename, and (I strongly recommend to select a passphrase)

ssh-keygen

– Copy public key from ~/.ssh/id_rsa.pub to your server location (in the example below 10.20.30.40)

ssh-copy-id -i ~/.ssh/id_rsa.pub username@10.20.30.40

– If you didn’t specify a passphrase, at this point you will be able to ssh to your server.

If you add a passphrase

You will need one more extra step to store the passphrase in the keychain so you won’t be asked to type it over and over again.
– Create a file in ~/.ssh/config
– Add these contents

Host *
   AddKeysToAgent yes
   UseKeychain yes

Now once you enter the passphrase, OS X wont ask you again.

Debugging SSH issues on OS X

Most common problems could be:

Directory permissions

directory permission permission code
/Users/[usename] 755 rwxr-xr-x
/Users/[usename]/.ssh 700 rwx——
/Users/[usename]/.ssh/id_rsa 600 rw——-

Modifications in SSH config files /etc/ssh/sshd_config

How to debug 

On the server:

– You could run another SSH process on different port, and monitor console log.

$ sudo /usr/sbin/sshd -d -p 4444

The client:

– Connect to the newly instantiated SSH process on port 4444 (-p 4444) with -v (verbose) option and monitor the log.

ssh -v -p 4444 tmux@10.0.1.4 -i ~/.ssh/tmux_ssh

Once you find the issue and fix it, you could restart the SSH server with the following commands:

$ sudo launchctl unload /System/Library/LaunchDaemons/ssh.plist
$ sudo launchctl load /System/Library/LaunchDaemons/ssh.plist

Understanding Java Script Map, Filter and Reduce

 

The array

// The array
var people = [
{name: 'john', age:24, weight: 84},
{name: 'tim', age:34, weight: 76},
{name: 'Jack', age:26, weight: 65},
];

MAP


// map - use it if I want to do the same operation for each element in the array
// and return the same amount of items in the array.
// array.map(
// current item of the array,
// curent index,
// the entire array
// );
var result = people.map((person, index, persons) => {
return `${person.name}: ${person.age}`;
});
console.log('map', result);

FILTER


// filter - return only items that match certain criteria.
// array.filter(
// current item of the array,
// curent index,
// the entire array
// )

var result = people.filter((person, index, persons) => {
return person.age < 30;
});
console.log('filter', result);

 

REDUCE


// reduce - return something completely new after iteration through each element
// in the array
// array.reduce(
// current value of the end value,
// current item,
// curent index,
// the entire array
// )
var combined_weight = people.reduce((weight, person, index, persons) => {
return weight += person.weight;
}, 0);
console.log('reduce', combined_weight);

Multiple asynchronous http calls using curl

If you ever needed to fetch data from multiple sources on the backend, probably you already explored the benefits of using curl multi fetch.
In the examples below, I’m using curl to fetch data from two urls, which simply have

<?php
sleep(3);
echo "Content 1 ...";

 intentionally to delay the content generation so we could see the difference between regular http call and asynchronous curl multi fetch.

http://toni-develops.com/sandbox/examples/php/curl-multi-fetch/fetched-content/fetch1.php
http://toni-develops.com/sandbox/examples/php/curl-multi-fetch/fetched-content/fetch2.php


<?php

$t1 = microtime(true);

function fetchContent($Url) {
// is cURL installed yet?
if (!function_exists('curl_init')){
die('Sorry cURL is not installed!');
}

// OK cool - then let's create a new cURL resource handle
$ch = curl_init();

// Now set some options (most are optional)

// Set URL to download
curl_setopt($ch, CURLOPT_URL, $Url);

// Set a referer
curl_setopt($ch, CURLOPT_REFERER, "http://www.example.org/yay.htm");

// User agent
curl_setopt($ch, CURLOPT_USERAGENT, "MozillaXYZ/1.0");

// Include header in result? (0 = yes, 1 = no)
curl_setopt($ch, CURLOPT_HEADER, 0);

// Should cURL return or print out the data? (true = return, false = print)
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// Timeout in seconds
curl_setopt($ch, CURLOPT_TIMEOUT, 10);

//Proxy if needed
//curl_setopt($ch, CURLOPT_PROXY, "64.210.197.20:80");

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);

// Download the given URL, and return output
$output = curl_exec($ch);

// Close the cURL resource, and free system resources
curl_close($ch);
return $output;
}

echo "<textarea style='width:100%;height:40%'>";
echo fetchContent('http://toni-develops.com/sandbox/examples/php/curl-multi-fetch/fetched-content/fetch1.php');
echo "</textarea>";

echo '<hr>';

echo "<textarea style='width:100%;height:40%'>";
echo fetchContent('http://toni-develops.com/sandbox/examples/php/curl-multi-fetch/fetched-content/fetch2.php');
echo "</textarea>";

echo 'fetched for: ' . (microtime(true) - $t1) . "\n";

 

<?php

// is cURL installed yet?
if (!function_exists('curl_init')){
die('Sorry cURL is not installed!');
}

$ch = array();
$mh = curl_multi_init();
$total = 100;

$t1 = microtime(true);

$URLs = array( "http://toni-develops.com/sandbox/examples/php/curl-multi-fetch/fetched-content/fetch2.php",
"http://toni-develops.com/sandbox/examples/php/curl-multi-fetch/fetched-content/fetch2.php");

$i = 0;
foreach($URLs as $url) {
$ch[$i] = curl_init();
curl_setopt($ch[$i], CURLOPT_URL, $url);
curl_setopt($ch[$i], CURLOPT_HEADER, 0);
curl_setopt($ch[$i], CURLOPT_RETURNTRANSFER, true);

curl_multi_add_handle($mh, $ch[$i]);
$i ++;
}

$active = null;
do {
$mrc = curl_multi_exec($mh, $active);
//usleep(100); // Maybe needed to limit CPU load (See P.S.)
} while ($active);

$content = array();

$i = 0;
foreach ($ch AS $i => $c) {
$content[$i] = curl_multi_getcontent($c);
curl_multi_remove_handle($mh, $c);
}

curl_multi_close($mh);

echo "<textarea style='width:100%;height:40%'>";
echo $content[0];
echo "</textarea>";

echo '<hr>';

echo "<textarea style='width:100%;height:40%'>";
echo $content[1];
echo "</textarea>";

echo 'fetched for: ' . (microtime(true) - $t1) . "\n";

 

So this way we are reducing the amount of time to fetch data from multiple URLs to the amount needed to complete the longest transaction.