architecture, workflow, CPU and distributed systems

dataflow processing model. In a dataflow model, computation is expressed as a directed graph of operators that transform inputs into outputs by applying operators such as transforms, aggregations, windowing, filtering or joins.

https://news.ycombinator.com/item?id=11796253

https://habrahabr.ru/post/280051/ Analytic engine RDF Apache Jena

http://www.archimatetool.com/

https://github.com/eljefe6a/pipeline Kafka+Spark+Dashboard

http://engineeringblog.yelp.com/2016/07/billions-of-messages-a-day-yelps-real-time-data-pipeline.html

https://engineering.linkedin.com/blog/2016/06/stream-processing-hard-problems-part-1-killing-lambda

https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102

http://martinfowler.com/articles/collection-pipeline/

http://blog.scalyr.com/2014/05/searching-20-gbsec-systems-engineering-before-algorithms/

http://www.firatatagun.com/blog/2016/01/09/analysis-of-software-architectures/

https://habrahabr.ru/post/276593/

https://habrahabr.ru/company/it-grad/blog/276297/

https://habrahabr.ru/post/276527/

https://habrahabr.ru/post/276613/

https://news.ycombinator.com/item?id=11464689 structure events

http://staltz.com/unidirectional-user-interface-architectures.html

http://martinfowler.com/eaaCatalog/

https://martin.kleppmann.com/2015/05/11/please-stop-calling-databases-cp-or-ap.html

https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

http://www.lucemorker.com/blog/introduction-to-software-architecture

http://habrahabr.ru/company/badoo/blog/272639/ Real Time Analytics

https://medium.com/@henridf/building-a-streaming-analytics-data-stack-ea0641048661

http://shop.oreilly.com/product/0636920041931.do

https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101

http://radar.oreilly.com/2014/07/questioning-the-lambda-architecture.html

http://www.oreilly.com/pub/e/3541

http://habrahabr.ru/post/271883/

http://habrahabr.ru/post/269589/

http://habrahabr.ru/company/custis/blog/265785/

http://elastisys.com/2015/09/10/scalability-design-principles/

http://highload.guide/

http://habrahabr.ru/company/oleg-bunin/blog/269119/

http://highscalability.com/blog/2015/11/9/a-360-degree-view-of-the-entire-netflix-stack.html

https://www.youtube.com/watch?v=R2kKmMyqTfc

https://github.com/Randommood/Strangeloop2015

http://habrahabr.ru/post/258145/

http://gameprogrammingpatterns.com./contents.html

http://habrahabr.ru/company/jugru/blog/261379/ Architecture

https://www.youtube.com/watch?v=DngAZyWMGR0

http://particular.net/s/fqvB6bdCIoVkQcJdWac

http://ookami86.github.io/event-sourcing-in-practice/ event sourcing

http://lambda-architecture.net/

https://spark-summit.org/2014/wp-content/uploads/2014/07/Lambda-Architecture-Jim-Scott..pdf

https://news.ycombinator.com/item?id=10118413

http://download.microsoft.com/documents/rus/msdn/%D1%80%D1%8B_%D0%BF%D1%80%D0%B8%D0%BB%D0%BE%D0%B6%D0%B5%D0%BD%D0%B8%D0%B9_%D0%BF%D0%BE%D0%BB%D0%BD%D0%B0%D1%8F_%D0%BA%D0%BD%D0%B8%D0%B3%D0%B0.pdf

https://raw.githubusercontent.com/druid-io/druid/master/publications/radstack/radstack.pdf

http://blog.confluent.io/2015/02/25/stream-data-platform-1/

http://www.oreilly.com/programming/free/files/software-architecture-patterns.pdf

https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

Composable analytics, DataFlow

https://github.com/common-workflow-language/common-workflow-language/wiki/Existing-Workflow-systems

https://www.michaelcho.me/article/data-pipelines-airflow-vs-pinball-vs-luigi

https://nifi.apache.org/docs/nifi-docs/html/overview.html

http://bytepawn.com/luigi-airflow-pinball.html

https://news.ycombinator.com/item?id=13761071

airflow

https://www.pandastrike.com/posts/20150914-airflow

http://tech.marksblogg.com/airflow-postgres-redis-forex.html

https://medium.com/handy-tech/airflow-tips-tricks-and-pitfalls-9ba53fba14eb#.biqtg26uw

https://wiki.openstack.org/wiki/TaskFlow

https://nifi.apache.org/

http://marcobonzanini.com/2015/10/24/building-data-pipelines-with-python-and-luigi/

http://engineering.bloomreach.com/briefly-python-dsl-scale-mapreduce-pipelines/

http://www.pythonsandbarracudas.com/blog/2015/11/22/developing-a-computational-pipeline-using-the-asyncio-module-in-python-3

https://medium.com/@jut_inc/why-devops-needs-dataflow-analytics-cd0cacaeeedd

https://medium.com/@henridf/building-a-streaming-analytics-data-stack-ea0641048661

https://composableanalytics.com/

http://paginas.fe.up.pt/~prodei/dsie12/papers/paper_17.pdf

https://www.youtube.com/watch?v=3oaelUXh7sE

http://blog.arkency.com/2015/03/why-use-event-sourcing/

http://www.infoq.com/minibooks/domain-driven-design-quickly

http://habrahabr.ru/company/suplbiz/blog/249873/

http://server.dzone.com/articles/designing-data-architecture-0

http://vitalflux.com/data-handled-linkedin-com/

https://www.youtube.com/watch?v=P68zXJ_ACCE

https://github.com/PharkMillups/beautiful-docs/blob/master/README.md

http://pages.cs.wisc.edu/~remzi/OSTEP/ OS free book

http://www.aosabook.org/en/index.html Book - Architecture of Open Source Applications

http://habrahabr.ru/post/232881/

http://highscalability.com/blog/2014/1/20/8-ways-stardog-made-its-database-insanely-scalable.html

https://news.ycombinator.com/item?id=8569930

http://gopulsar.io/ Real-time streaming analytics

Microservices

http://codingsans.com/blog/microservice-architecture-best-practices

https://habrahabr.ru/company/2gis/blog/343628/

http://nginx.com/blog/adopting-microservices-at-netflix-lessons-for-team-and-process-design/

https://herbertograca.com/2017/01/26/microservices-architecture/

https://habrahabr.ru/company/mailru/blog/320962/

http://martinfowler.com/articles/microservices.html

https://cloudncode.blog/2016/07/22/msa-getting-started/

http://microservices.io/patterns/

http://engineeringblog.yelp.com/2015/03/using-services-to-break-down-monoliths.html

http://eax.me/micro-service-architecture/

http://philcalcado.com/2015/09/08/how_we_ended_up_with_microservices.html

Important book about design

http://blog.confluent.io/2015/01/29/making-sense-of-stream-processing/

http://www.amazon.com/Implementing-Domain-Driven-Design-Vaughn-Vernon/dp/0321834577

http://techblog.netflix.com/2014/11/introducing-dynomite.html

http://www.informit.com/articles/article.aspx?p=1963779

Clean Architecture with Python

http://rhodesmill.org/brandon/slides/2013-10-pyconie/

https://habrahabr.ru/post/319126/

OS memory/file management

http://habrahabr.ru/post/209202/

http://habrahabr.ru/post/209758/ Scaling

DataFlow

http://dataflowbook.com/cms/?p=420

https://news.ycombinator.com/item?id=8867584

https://medium.com/data-engineering/don-t-orchestrate-when-you-can-compose-daaf5cce1f0e#.yyz68ibl1

http://highscalability.com/

http://www.infoq.com/presentations/Architecture-Uncertainty

http://www.ibm.com/developerworks/library/bd-archpatterns5/index.html

http://bamboo.io/docs/index.html

DataFlow and WorkFlow

http://blog.factual.com/introducing-drake-a-kind-of-make-for-data

https://news.ycombinator.com/item?id=12136011

http://www.commonwl.org/

https://news.ycombinator.com/item?id=11632205

http://bytepawn.com/luigi-airflow-pinball.html

https://github.com/tinkerpop/pipes/wiki

http://www.pythonect.org/

https://code.google.com/p/ruffus/

https://pypi.python.org/pypi/DAGPype

https://github.com/spotify/luigi

http://edn.embarcadero.com/article/30360 C++ pipeline

http://billwadge.wordpress.com/2011/04/27/lucid-the-dataflow-language/

http://dataflowbook.com/cms/?p=389

http://cacm.acm.org/blogs/blog-cacm/169199-data-science-workflow-overview-and-challenges/fulltext

http://engineering.klout.com/2013/08/scoozie-creating-big-data-workflows-at-klout/

https://github.com/mdipierro/workflow python

Dataflow and Reactive Programming Systems e-book http://dataflowbook.com/cms/?p=342

http://www.drdobbs.com/database/dataflow-programming-handling-huge-data/231400148?pgno=1

http://www.dancinghacker.com/code/dataflow/dataflow/introduction.html

ETL /Workflow

http://slott-softwarearchitect.blogspot.com/2010/11/data-mapping-and-conversion-tools-sigh.html

http://www.manageability.org/blog/stuff/workflow_in_java

http://nosql.mypopescu.com/

https://learn-cs.pbworks.com/w/page/15774601/FrontPage

http://techblog.netflix.com/search/label/high%20volume%20distributed%20systems

http://lethain.com/introduction-to-architecting-systems-for-scale/

http://reverbrain.com/grape/

http://blog.treasure-data.com/post/16034997056/enabling-facebooks-log-infrastructure-with-fluentd

https://news.ycombinator.com/item?id=6946491 Architectures

http://doubleclix.wordpress.com/

Масштабирование нагрузки web-приложений

http://habrahabr.ru/post/113992/

http://habrahabr.ru/company/evernote/blog/120319/ Evernotes

http://habrahabr.ru/post/135364/ Uникальный ключ в условиях распределенной

http://www.josuttis.com/soa-in-practice/index.html Book

Pattern-Oriented Software Architecture For Dummies. Robert Hanmer (Book)

Kirk Hausman, "Sustainable Enterprise Architecture" (Book)

Fon Silvers, "Data Warehouse Designs: Achieving ROI with Market Basket Analysis and Time Variance"

http://www.reddit.com/r/compsci/comments/1idf4n/good_places_to_learn_software_architecture/

http://www.lulu.com/shop/amy-brown-and-greg-wilson/the-architecture-of-open-source-applications-volume-ii/ebook/product-20133259.html Book

http://www.systemswemake.com/

http://www.infoq.com/author/Rich-Hickey

Stream Processing

http://strata.oreilly.com/2013/09/stream-processing-and-mining-just-got-more-interesting.html

http://highlyscalable.wordpress.com/2013/08/20/in-stream-big-data-processing/

http://habrahabr.ru/company/mailru/blog/136288/ Tarantool

Comparison of Approaches to Large-Scale Data Analysis

http://www.cse.nd.edu/~dthain/courses/cse598z/spring2010/benchmarks-sigmod09.pdf

http://www.highload.ru/2012/abstracts/430.html Аналитическая инфраструктура оптимизации рекламной сети

http://concord.io/why/

http://www.highload.ru/2012/abstracts/393.html

http://habrahabr.ru/post/190740/ HP Vertica

http://habrahabr.ru/company/lifestreet/blog/147254/

http://avaxhome.ws/ebooks/programming_development/1119963990Pattern.html

http://habrahabr.ru/post/185636/

http://blog.mashape.com/post/60820526317/list-of-40-tutorials-on-how-to-create-an-api

Google's BigTable: http://www.computerra.ru/83598/bigtable/

http://williamedwardscoder.tumblr.com/post/16516763725/how-i-got-massively-faster-db-with-async-batching

http://williamedwardscoder.tumblr.com/post/53349845356/scaling-my-server

http://queue.acm.org/detail.cfm?id=2513149 NUMA

http://mrjoes.github.io/2013/06/21/python-realtime.html

http://blog.filmgrainapp.com/2013/06/25/how-we-built-filmgrain-part-1-of-2/

http://habrahabr.ru/company/badoo/blog/185220/

http://habrahabr.ru/company/badoo/blog/185596/

http://highscalability.com/

http://highscalability.com/blog/2013/7/8/the-architecture-twitter-uses-to-deal-with-150m-active-users.html

https://gist.github.com/ragingwind/5840075

http://blog.softartisans.com/2012/02/13/bostons-big-datascape-part-3/

http://ayedo.github.io/netty/2013/06/19/what-is-netty.html

http://mechanical-sympathy.blogspot.com/2013/02/cpu-cache-flushing-fallacy.html

http://marakana.com/s/post/1493/big_data_scala_with_spark_mark_hamstra_video

http://mechanical-sympathy.blogspot.com/

http://eferm.com/implementing-durability-for-in-memory-databases-on-ssds/

http://www.infoq.com/

https://www.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html

Disruptor

http://lmax-exchange.github.io/disruptor/

http://martinfowler.com/articles/lmax.htm

http://www.slideshare.net/trishagee/introduction-to-the-disruptor?from=new_upload_email

ACM magazine

https://queue.acm.org/detail.cfm?id=2488549

https://queue.acm.org/detail.cfm?id=2492433 NonBlocking Algo and Multicore Programming

http://shark.cs.berkeley.edu/ Shark/Spark

http://habrahabr.ru/post/182164/ Dryad

http://hackingdistributed.com/2013/06/17/hyperleveldb/

GIS

http://www.openstreetmap.org/

http://functionspace.org/topic/31/Google-map-searching-a-path-from-place-A-to-place-B

http://habrahabr.ru/post/182998/ Spatial hashing

http://habrahabr.ru/post/183288/

http://habrahabr.ru/post/186672/

https://news.ycombinator.com/item?id=5875262

http://rjw57.github.io/blog/2013/06/19/real-time-traffic-data/

http://www.qgis.org/

pdf tools

http://www.pdfsam.org/

Memory

https://habrahabr.ru/post/316520/

http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory

https://habrahabr.ru/company/intel/blog/283196/

https://habrahabr.ru/company/1cloud/blog/276217/ virtual memory

https://habrahabr.ru/company/nixsolutions/blog/277759/

http://habrahabr.ru/company/smart_soft/blog/185226/

http://habrahabr.ru/company/smart_soft/blog/226315/

http://habrahabr.ru/company/smart_soft/blog/228937/

http://people.redhat.com/drepper/cpumemory.pdf

http://www.infoq.com/presentations/click-crash-course-modern-hardware

https://lwn.net/Articles/250967/?repost=1

https://techtalk.intersec.com/2013/07/memory-part-1-memory-types/

https://techtalk.intersec.com/2013/07/memory-part-2-understanding-process-memory/

https://techtalk.intersec.com/2013/08/memory-part-3-managing-memory/

https://techtalk.intersec.com/2013/10/memory-part-4-intersecs-custom-allocators/

https://techtalk.intersec.com/2014/09/memory-part-6-optimizing-the-fifo-and-stack-allocators/

http://people.redhat.com/drepper/

http://code.google.com/edu/languages/

http://personal.inet.fi/koti/egaga/recommendations/index.html

http://en.wikipedia.org/wiki/List_of_numerical_analysis_software/

Hardware

clang++ -stdlib=libc++ poly.cpp

http://www.lighterra.com/papers/modernmicroprocessors/

http://lolengine.net/blog/2011/9/17/playing-with-the-cpu-pipeline

http://habrahabr.ru/post/182002/ x86

http://habrahabr.ru/company/neobit/blog/181626/ x86

FPGA

https://news.ycombinator.com/item?id=8375111

http://www.yosefk.com/blog/how-fpgas-work-and-why-youll-buy-one.html FGPA

https://news.ycombinator.com/item?id=5895672

GPU

http://www.cs.virginia.edu/~gfx/papers/paper.php?paper_id=59

Page updated

Google Sites

Report abuse