Web based IDE with React & Microsoft Monaco Editor

FAAS and Web based IDE

Certain product customization needs are too complex. These can not be handled by configuration alone. These customizations need to be encapsulated in some Functions. These Functions are then executed in a secured sandbox against certain events. The product now houses a Function as a Service (FAAS) platform

A product that has a Function as a Service (FAAS) embedded in them, would need a web based IDE to allow users to write and execute Functions.

Sounds familiar, AWS lambda is a classic example. AWS lambda is supported by an online web based IDE named AWS cloud9 IDE.

Web based IDE Solutions

In my search for a viable web based IDE for coding Functions, I came across Microsoft Monaco Editor.

There are few IDE solutions, I came across but I choose Microsoft Monaco Editor as it seems to satisfy my needs for Typescript language. It’s no surprise, since both the editor and the language is backed by Microsoft.

Note — Other IDE solutions I came across where were Ace and CodeMirror.

Features of Web based IDE Solutions

Following where the features I was targeting for my IDE

  • Support for Typescript Language

  • Support for import statements in the code with auto completion

  • Auto Completion to be backed by Typescript Definitions

  • Display Errors when Code is not compatible with Typescript Language

  • Display Errors when Code is not adhering to Typescript Type Definitions

screenshot.png

React Monaco Editor

Demo — https://rohitghatol.github.io/react-monaco-editor-example/

Repo — https://github.com/rohitghatol/react-monaco-editor-example

Step 1. Create React App and Eject React Scripts

$>npm i create-react-app -g
$>create-react-app 
$>cd app
$>npm eject

Step 2. Install react-monaco-editor

$> npm i react-monaco-editor -S
$> npm i monaco-editor-webpack-plugin -D

Edit app/config/webpack.config.js file

1. Add Import at the top

const MonacoWebpackPlugin = require('monaco-editor-webpack-plugin');

2. Add Plugin in the plugin space

new MonacoWebpackPlugin(),

Refer to webpack.config.js

Step 3. Write Simple Typescript Editor using react-monaco-editor

Refer to the following code. The key things to note here are

  1. Import of react-monaco-editor

  2. Usage of <MonacoEditor/> tag with attributes “language”, “value” and “theme”

  3. The Code which has the Interface definition and a const value using the Interface Definition

// Define Typescript Interface Employee
interface Employee {
    firstName: String;
    lastName: String;
    contractor?: Boolean;
}

// Use Typescript Interface Employee. 
// This should show you an error on john 
// as required attribute lastName is missing
const john:Employee = {
    firstName:"John",
    // lastName:"Smith"
    // contractor:true
}

SimpleTypescriptEditor.jsx

import React,{Component} from 'react';
import MonacoEditor from 'react-monaco-editor'

const code =
`

// Define Typescript Interface Employee
interface Employee {
    firstName: String;
    lastName: String;
    contractor?: Boolean;
}

// Use Typescript Interface Employee. 
// This should show you an error on john 
// as required attribute lastName is missing
const john:Employee = {
    firstName:"John",
    // lastName:"Smith"
    // contractor:true
}

`
export class SimpleTypescriptEditor extends Component {


    constructor(props){
        super(props);
        this.state = {
            code
        }
    }

    onChange(newValue, e) {
        // console.log('onChange', newValue, e);
    }

    render() {
        return (
            <MonacoEditor
                width="600"
                height="800"
                language="typescript"
                theme="vs-dark"
                defaultValue=''
                value={this.state.code}
                onChange={this.onChange}
            />
        )
    }
}

Demo — https://rohitghatol.github.io/react-monaco-editor-example/

Advanced Use case

See the example mentioned below. In this piece of code, we have imports from modules named “lambda”, “models” and “ramda”.

import {Event, Context} from "lambda";
import {Item, Result} from "models"
import * as R from "ramda";
 
export const lambda = async (event:Event, context:Context): Promise<Result[]> => {
    
    const result:Result[] = R.map((input:Item) => ({
        id: input.id,
        name: input.name,
        value: input.value
    }),event.input);
    
    return result;
}

In order to make this work we have to understand this, we have to visualize a folder structure

src/
   main.tsx
   node_modules/
       models/
           index.d.ts
       lambda/
           index.d.ts
       ramda/
           index.d.ts
src/components/types
   index.js
   models.js
   lamdba.js
   ramda.js
//index.js
import {content as modelContent} from './models';
import {content as lambdaContent} from './lambda';
import {content as ramdaContent} from './ramda';

export const files = {
    "models/index.d.ts": modelContent,
    "ramda/index.d.ts": ramdaContent,
    "lambda/index.d.ts": lambdaContent,
};
//models.js
export const content =
`
export interface Item {
   id: String;
   name: String;
   value: String;
}

export interface Result {
   id: String;
   name: String;
   value: String;
}
`
//lambda.js
export const content =
`
import {Item} from 'models';

export interface Event {
    input: Item[]
 }

export interface Context {
   
}
`

Lets make necessary changes to our SimpleTypescript editor to make it support imports.

  • We need to add a callback handler called editorWillMount. In this callback function we will inject the type definitions file in monaco editor

editorWillMount(monaco) {

        // validation settings
        monaco.languages.typescript.typescriptDefaults.setDiagnosticsOptions({
            noSemanticValidation: false,
            noSyntaxValidation: false
        });

        // compiler options
        monaco.languages.typescript.typescriptDefaults.setCompilerOptions({
            target: monaco.languages.typescript.ScriptTarget.ES6,
            allowNonTsExtensions: true
        });

        for (const fileName in files) {
            const fakePath = `file:///node_modules/@types/${fileName}`;

            monaco.languages.typescript.typescriptDefaults.addExtraLib(
                files[fileName],
                fakePath
            );
        }


    }

In the options section we will provide a model. This model will ensure that the only created once.

const options = {
            selectOnLineNumbers: true,
            model: monaco.editor.getModel(Uri.parse("file:///main.tsx"))
                ||
                monaco.editor.createModel(code, "typescript", monaco.Uri.parse("file:///main.tsx"))
        }
<MonacoEditor
                width="800"
                height="800"
                language="typescript"
                theme="vs-dark"
                defaultValue=''
                value={this.state.code}
                onChange={this.onChange}
                editorWillMount={this.editorWillMount}
                editorDidMount={this.editorDidMount}
                options={options}
            />

Finally the complete code looks like this.


//AdvancedTypescriptEditor.jsx
import React,{Component} from 'react';
import MonacoEditor from 'react-monaco-editor'
import * as monaco from 'monaco-editor/esm/vs/editor/editor.api';
import {Uri} from 'monaco-editor/esm/vs/editor/editor.api';

import {files} from './typings';

const code =
`
//--------------------------
// Contents of "models"
//--------------------------
//
// export interface Item {
//    id: String;
//    name: String;
//    value: String;
// }
//
// export interface Result {
//    id: String;
//    name: String;
//    value: String;
// }

//--------------------------
// Contents of "lambda"
//--------------------------
//
// import {Item} from 'models';
//
// export interface Event {
//     input: Item[]
// }
//
// export interface Context {
//  
// }


import {Event, Context} from "lambda";
import {Item, Result} from "models"
import * as R from "ramda";
 
export const lambda = async (event:Event, context:Context): Promise<Result[]> => {
    
    const result:Result[] = R.map((input:Item) => ({
        id: input.id,
        name: input.name,
        value: input.value
    }),event.input);
    
    return result;
}




`
export class AdvancedTypescriptEditor extends Component {


    constructor(props){
        super(props);
        this.state = {
            code
        }
    }

    onChange(newValue, e) {
        // console.log('onChange', newValue, e);
    }

    editorWillMount(monaco) {

        // validation settings
        monaco.languages.typescript.typescriptDefaults.setDiagnosticsOptions({
            noSemanticValidation: false,
            noSyntaxValidation: false
        });

        // compiler options
        monaco.languages.typescript.typescriptDefaults.setCompilerOptions({
            target: monaco.languages.typescript.ScriptTarget.ES6,
            allowNonTsExtensions: true
        });

        for (const fileName in files) {
            const fakePath = `file:///node_modules/@types/${fileName}`;

            monaco.languages.typescript.typescriptDefaults.addExtraLib(
                files[fileName],
                fakePath
            );
        }


    }



    editorDidMount(editor, monaco) {
        editor.focus();
    }

    render() {
        const options = {
            selectOnLineNumbers: true,
            model: monaco.editor.getModel(Uri.parse("file:///main.tsx"))
                ||
                monaco.editor.createModel(code, "typescript", monaco.Uri.parse("file:///main.tsx"))
        }
        return (
            <MonacoEditor
                width="800"
                height="800"
                language="typescript"
                theme="vs-dark"
                defaultValue=''
                value={this.state.code}
                onChange={this.onChange}
                editorWillMount={this.editorWillMount}
                editorDidMount={this.editorDidMount}
                options={options}
            />
        )
    }
}

The Editor looks as follows


screenshot-002.png

Setting up Hue on Amazon AWS

Amazon AWS now supports Hue. This blog explains the steps required to set up and access Hue on Amazon EMR Step 1 - Create Amazon EMR Cluster with Hue application selected

Hue-1
Hue-1
Hue-2
Hue-2
Hue-3
Hue-3

Step 2 - Once you start the Cluster, click on "Enable Web Connection", setup ssh tunnel and web proxy to access Hue

Hue-4
Hue-4
Hue-5
Hue-5
Hue-11
Hue-11
Hue-6
Hue-6
Hue-7
Hue-7
Hue-8
Hue-8

Step 3 - After installing and configuring FoxyProxy access the hue web and set your login and password. Now access Hue on EMR

Hue-9
Hue-9
Hue-10
Hue-10

Step 4 -  Run Hive Query

Choose the AWS Sample: ELB access logs project and then Execute the Hive query

Hive Query
Hive Query
Hive Query Result
Hive Query Result

3D Presentation Framework built on Polymer.js (Web Components)

Introduction

KinoScribe is a 3D Presentation framework built using Polymer.js (Google's Web Component's Library). Using KinoScribe you can create 3D Presentations in which you can layout your side in 3D using

  • Custom Layout - You define all 3D Coordinates, Rotations etc
  • Preset Layouts - Where you use one of the Preset Layouts like "Left to Right", "Top to Bottom", "Box", "Spiral", "Album"

Video Demo

https://www.youtube.com/watch?v=l4Dauw3oJ1o

Home Page

Visit - http://www.kinoscribe.com for live demo

Code Examples

Visit - http://www.kinoscribe.com for live demo

Browser Compatibility

  • Chrome
  • Firefox

Yet to be tested on Safari and IE 10+

Usage

<!doctype html> <html> <head> <meta name="viewport" content="width=device-width, minimum-scale=1.0, initial-scale=1.0, user-scalable=yes"> <title>kino-slides Demo</title>

<script src="../platform/platform.js"></script> <link rel="import" href="kinoscribe.html"> <style> kino-slide .slide{ display: block; width: 800px; height: 600px; padding: 40px 60px; background-color: white; border: 1px solid rgba(0, 0, 0, .3); border-radius: 10px; box-shadow: 0 2px 6px rgba(0, 0, 0, .1); color: rgb(102, 102, 102); text-shadow: 0 2px 2px rgba(0, 0, 0, .1); font-family: 'Open Sans', Arial, sans-serif; font-size: 30px; line-height: 36px; letter-spacing: -1px; } </style> </head> <body unresolved>

<kino-pres> <kino-slide x="1000" type="transition" > <section class="slide"> Slide 1 </section> </kino-slide>

<kino-slide x="2500" z="-1000"> <section class="slide"> Slide 2 </section> </kino-slide>

<kino-slide x="4000" z="-2000" scale="3"> <section class="slide"> Slide 3 </section> </kino-slide>

<kino-slide x="5500" z="-3000" rotateZ="45"> <section class="slide"> Slide 4 </section> </kino-slide>

<kino-slide x="7000" rotateX="90" rotateY="90"> <section class="slide"> Slide 5 </section>

</kino-slide> <kino-slide x="8500" z="-1000" > <section class="slide"> Slide 6 </section> </kino-slide>

</body> </html>

Authors

  • Rohit Ghatol - @rohitghatol
  • Nikhil Walvekar - @walvekarnikhil

Install Sqoop on Amazon EMR

Overview

This is a detailed Tutorial on how to install Sqoop on Amazon EMR Cluster and import data from MySQL Database to Amazon S3 Bucket

Tutorial Video

https://www.youtube.com/watch?v=3YJwDJOyDE0

Prerequisites

Download the following files and upload them to your S3 Bucket

  1. Sqoop Binary - http://archive.apache.org/dist/sqoop/1.4.4/sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz
  2. MySQL JDBC Connector - http://dev.mysql.com/downloads/connector/j/5.1.html

Install-Sqoop.sh

#!/bin/bash

cd /home/hadoop
hadoop fs -copyToLocal s3://synerzip-sqoop-scripts/sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz
tar -xzf sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz
hadoop fs -copyToLocal s3://synerzip-sqoop-scripts/mysql-connector-java-5.1.33.tar.gz mysql-connector-java-5.1.33.tar.gz
tar -xzf mysql-connector-java-5.1.33.tar.gz
cp mysql-connector-java-5.1.33/mysql-connector-java-5.1.33-bin.jar sqoop-1.4.4.bin__hadoop-2.0.4-alpha/lib/

Ensure no CRLF characters are present in the file

Sqoop-Import-all.sh

 

#!/bin/bash

cd /home/hadoop/sqoop-1.4.4.bin__hadoop-2.0.4-alpha/bin

./sqoop import --connect jdbc:mysql://db.c5zzejm1gdnx.us-west-1.rds.amazonaws.com/test --username root --password password
--table User_Profile --target-dir s3://synerzip-imported-data/User_Profile-`date +"%m-%d-%y_%T"`

Ensure no CRLF characters are present in the file

 

Steps

Step 1

Slide01

Step 2

Slide02

Step 3

Slide03

Step 4

Slide04

Step 5

Slide05

Step 6

Slide06

Step 7

Slide07

Step 8

Slide08

Step 9

Slide09

Step 10

Slide10

Step 11

Slide11

Step 12

Slide12

Step 13

Slide13

Step 14

Slide14