2019-07-24 JUL
Congrats to India :-)
Hackathons
GCP
Let's review a bit of GCP every session so we explore the Cloud execution model systematically. Starting with reviewing Bhaumik's discussion of FaaS vs Apps:
Bhaumik Patel @bhaumik55231 Jul 17 11:55
A really cool video on Cloud functions and App Engine, how they work behind the schene and what are the best practices : - https://www.youtube.com/watch?v=MBBQ6P3GauY :
- Counterpoint: https://www.theregister.co.uk/2018/12/19/serverless_computing_study/
- "The UC Berkeley crew contends that FaaS is an architectural anti-pattern that overlooks the realities of latency, bandwidth, and cost.... [T]he boffins conducted several tests. One involved training a machine learning model on Lambda and doing the same on AWS EC2. The result: Lambda was 21x slower and 7.3x more expensive than EC2."
> Nicole mentioned feeling restricted by SQL-only queries in GCP's BigQuery - there's also CosmosDB, from Azure
Note CosmosDB equivalent in GCP is FireStore, not BigQuery. Huge fan of CosmosDB here (Jonas), here's a podcast where its lead dev discusses it. I'd suggest FaaS and FireStore as our minimum FAIR dev env.
Data/File Structures - Collection/[Document|Collection] model
Nicole SAS data maps
they have an interesting parallelism with GraphQL Schemas, which, SAS supports https://blogs.sas.com/content/tag/graphql
- Note both scalability and governance rationale: https://youtu.be/lW7DWV2jST0.
- And how event driven FaaS allows data structure to evolve (discuss evolvability).
Discussion points
- non-zero idle - what does it mean, what does it cost, how does it scale?
- open client - the bleeding edge of the data ecosystem
- why APIs are the thin-middle of data commons.
TensorFlow + tracking each-other
Marie has made good progress with tfjs, you can follow them at https://github.com/cloud4bio/marie. Note how both the application and the chatter are available. This may be a good model to follow for other projects. Note how the level of exposure, and scrutiny, is so much higher than what you may be confortable with.
CLIs
Contribution to recurring conversation by Eric
https://www.twilio.com/blog/how-to-build-a-cli-with-node-js