Skip to content

Latest commit

 

History

History
117 lines (102 loc) · 9.36 KB

README.md

File metadata and controls

117 lines (102 loc) · 9.36 KB

is-it-open-restaurants

Simple Web App to query Restaurant Opening Hours (from a given csv)
Task Requirements: https://gist.github.com/seahyc/d013a8f8f1c1be52513cf7b77cce6e81

How to use this repo

This readme on main describes the general updates, TODOS for frontend/backend and updates that can't be quantified as code this repo(deployment settings, setting up of db/API gateway/lambda fns in aws) for all branches.

Keeping master branch as the latest working full stack deployment, so I pull-req from a dev branch to master once a version is complete.

I started off designing my architecture as reactjs -> awsAPIgateway -> lambda function -> AWS Elasticsearch, but faced multiple issues including CORS and lambda fn deployment. So I went for a simpler stack using reactjs --> jsonbin.io (json store), but eventually got my initial design(on branch dev-jquery-aws-elasticsearch) working!

While my final deployment is simply reactjs on AWS Amplify --> jsonbin.io REST API --> jsonbin.io db, I can explain my initial architectural decisions of using Elasticsearch, API Gateway and serverless component of lambda function.

What's Deployed and Where?

Branch Frontend Backend Db
reactjs-jsonbinio ReactJS on AWS Amplify: https://main.d2rwu09ypt5g13.amplifyapp.com/ - (didn't need express) jsonbin.io
(experimental) jquery-aws-elasticsearch plain html and jquery's HTTP methods: https://dev-jquery-aws-elasticsearch.dxbdt2o26wbkw.amplifyapp.com/ AWS API Gateway (success), lambda function (success). Update after Consult with AWS Technical Writer Liz Synder AWS ElasticSearch (success) UPDATE: removed this db instance because instance hours was too expensive, contact lzh.carey@gmail.com for demo

Latest Notes

  • main branch code currently based on branch dev-reactjs-jsonbin
git clone https://github.com/careylzh/is-it-open-restaurants.git
cd client
npm i
npm start
  • currently prototyping collections page on dev-reactjs-jsonbin branch, only implemented non-func UI for this feature
  • ERR_NAME_NOT_RESOLVED surfaced because forgot to update new awsApiGatewayEndpoint (main one, not the specific ones for GET, POST etc)
  • CORs issue fixed by deleting API Gateway and regenerating but checking the 5xx and 4xx tickboxes. Verified using http://client.cors-api.appspot.com/client
  • simplified stack because found jsonbin.io, a JSON store which has several benefits:
    • auto-configs CORS-settings
    • simple UI for organising collections of documents
    • can add schema ltr if needed.
  • AWS Amplify does snapshot testing on various mobile devices for you so no need npm test for snapshots wew [add photo]
  • Elasticsearch noSQL API syntax: using "fields" in lambda fn references the fields in under "_source:" in each document in Elasticsearch
  • Lambda fn: edited "queryStringParameters" with "s" and now there's no keyError

dev-reactjs-jsonbin specific notes:

  • search by name and display openingHours done.
  • To add datepicker and integrate datepicking search algo in js (semantic versioning should be 0.9.6 now - when tabnav is complete(collections page) 0.9.7, 0.9.8 favouriting function, 0.9.9 when POST queries to json bin are done, 1.0.0 when passing date from datepicker UI to SearchBarResults component)

Separate Adventures:

  • Successfully parsed raw data and populated AWS Elasticsearch(abbreviated ES, not ECMA) DB with Kibana Console in required BULK format
  • Priortised Name Searching Speed so decided to use non-rel db over rel. Also corrobated w fact that Elasticsearch is industrial grade search eng/db. But number searching might be slow
  • Probably have to reformat openingHours field in elasticsearch restaurants for future usage/ease of access by new, unplanned features eg. new location tag in each json string
  • If API Gateway/Lambda Function is configured properly: Finetune Cloudwatch logs to throw exact 5xx/4xx error statuses.
  • Paused development for dev-jquery-aws-elasticsearch stack due to unresolved CORS setting despite setting "Access-Control-Allow-Origin" to '*' on AWS API gateway

The follow sections explain the initial technical decisions/progress revolving dev-jquery-aws-elasticsearch.

AWS Elasticsearch

[add photo for kibana console queries]

  • main index for all data 2214 data points: collection_of_shops
  • Kibana endpoint for testing (internally revealed) has login credentials that is differently set from aws IAM/secret key authentication

API Protection using AWS API Gateway Interface (probably similar to how nginx works?)

AWS Lambda: the Serverless, event-triggered faciltator

  • component that facilitates communication between API Gateway
  • talks to elasticsearch API to return documents of a particular index
  • could either write separate lambda fns for separate API calls from reactjs/jquery. Lambda functions will form the backend like how express.js routes data from frontend to db, vice versa
  • Lambda function deployed successfully
  • How to update lambda fn:
  • Can try using Serverless framework to deploy using CLI so no need keep uploading .zip of your LambdaFunction folder onto AWS console

TODO - General

  • requirement analysis, architecture planning for scalability
  • set up proj structure using npx/npm for client and server folders, helper functions
  • set up branches (main=live, dev-reactjs-jsonbin, dev-jquery-aws-elasticsearch)
  • deploy v0.9.4 from master branch
  • update deployment notes here w.r.t dev-jquery-aws-elasticsearch branch (alot of work)

TODO - frontend

reactjs-jsonbin

  • frontend wireframe in terms of html elements
  • implement reactjs search bar and native js REST API
  • implement tab navigation/drawer reactjs
  • implement reactjs datepicker and write found python data retrieval code for datepicker in js
  • css formatting reactjs (not a priority)
  • create reactjs Collections page for one user
  • implement jquery + plain js for search bar
  • implement jquery/plain js datepicker
  • (additional) disconnect from jsonbin db, find out how to connect reactjs to apiGatewayEndpoint

TODO - backend

AWS configs (additional explorations)

  • peform unit test on lambda function using AWS Lambda Web Testing Interface (used API Gateway Testing Interface instead

  • fix annoying CORS issue on API gateway/lambda function

  • implement lambda functions/API gateway methods for other CRUD functions (datesearch, create new indexes for collections for a particular user)

  • create new branch from explore using aws-elasticsearch-connector (run on separate node instance as reactjs), express to facilitate communication between reactJS and aws es db access

  • LAST-RESORT BY 17/4/21 5PM: implement Firebase/jsonbin.io db with reactjs front end if AWS Full Stack doesn't work