Talks
Next meeting - Talk27 - 20 June, 1.00pm AEST
Tech Talk 27 Service Mesh Overview and Anatomy
Coming up
Talk 27 - 20 June, 1.00pm AEST - Service Mesh Overview and Anatomy
This presentation explains how a Kubernetes cluster can use a service mesh to abstract the network away, resolving many of the challenges arising from talking to remote endpoints within a Kubernetes cluster. A service mesh is a tool that inserts security, observability, and reliability features to applications at the platform layer instead of the application layer. It controls the delivery of service requests to other services, performs load balancing, encrypts data, and discovers other services. Service mesh is not a network service for establishing connectivity between the microservices, instead, it has policies and controls that are applied on top of an existing network to govern how microservices interact. (Slides, Video)
Speaker
Ping Chen, I had the privilege of being a part of the ARDC Nectar core team for approximately two enriching years as senior DevOps engineer based at Monash Clayton campus. Before my tenure at ARDC, I gained valuable experience with various Telecommunication equipment vendors, where I contributed to the development of end-to-end carrier IP/MPLS networks and the establishment of automated, optimized, fault-tolerant on-prem Data Centres to accommodate EMS (Element Management Systems). In these roles, I offered architectural guidance, conducted thorough requirement analyses, crafted specifications, shared technical product insights, and conducted meticulous technical reviews. Within the dynamic realm of Telecommunications, critical thinking and adept problem-solving were indispensable skills. My academic background includes a master’s degree in enterprise architecture and a bachelor’s degree in electronic and communications engineering from Melbourne Uni.
Talk 26 - 24 April, 2.00pm AEST - Techniques and Software Framework for Extracting Metadata from Diverse Data Sources
Like many other kinds of government and research institutions, Australia's geological institutions house datasets for which there is a broad variation in the quality of metadata. This creates challenges for data providers and aggregators trying to maintain a certain standard of FAIR compliance across all their offerings.
For example, when the offering is very poor, more research and manual data entry may be required. For aggregators there is the problem of extracting metadata of a consistent standard from a wide variety of catalogue systems.
This talk will outline a software architecture and techniques that attempt to alleviate the burden of metadata collation and curation. The software is designed to homogenize metadata harvested from a variety of common metadata catalogue applications. For metadata-poor sources, extraction of metadata from associated technical reports using textual analysis and machine learning models is utilised. The limitations and viability of such techniques are discussed. At the end of the transformation process, ISO-compliant metadata records are created which are suitable for importing into a geo-network geospatial catalogue. (Slides, Video)
Speaker
Vincent Fazio is a software developer employed by CSIRO Mineral Resources to implement AuScope's future vision for Australian geoscience research. He has worked in a broad range of industries and research areas over the past 20 years including defence, telecommunications, hand-held devices and protein crystallography. His current interests include: implementing metadata standards, geospatial information systems, displaying 3D geospatial datasets, website development and open source software.
Talk 25 - 22 Feb, 2.00pm AEDT - Clean Air Research Data and Analysis Technology (CARDAT): A Data platform
Reliable data and decision support systems are critical for evidence-based environmental health surveillance and action in response to the climate change crisis. Efforts to acquire and harmonise environmental and health data to assess impacts of the range of environmental change are often inefficient. Analytic tools such as health impact assessment, burden of disease analysis, cost-benefit analysis and economic-energy models are often difficult to link and interpret. With support from NHMRC and ARDC we have developed the CARDAT platform for sharing both 1) data (i.e. health records and environmental exposures) and 2) analyses (i.e. code and models). This seminar will describe the design and implementation of the range of solutions we used to address the issues posed, applying the FAIR criteria to share health and environmental data/analysis, all while safeguarding confidentiality. The recent 'AirHealth' project within ARDC's 'Public Sector to Research Sector Bridges' program demonstrates CARDAT's utility by building collaborative links between key existing resources at the Australian Institute of Health and Welfare and the NHMRC Centre for Safe Air's air pollution modelling. Research data infrastructure such as CARDAT are urgently needed to enable integrated assessment of health impacts, costs and benefits of adaptation and mitigation interventions to respond to climate change. (Slides, Video)
Speaker
Dr Ivan Hanigan PhD, Director, WHO Collaborating Centre for Climate Change and Health Impact Assessment | Faculty of Health Science, Senior Lecturer in Climate Change and Health | School of Population Health
Talk 24 - 30 November, 2.00pm AEDT - Cross-platform GPU programming without the heartburn
Graphics processing units (GPUs) are an important component of high-performance computing and AI infrastructure, as they offer substantially higher performance and efficiency than general CPUs for many scientific workflows. However, programming for GPUs requires close attention to low-level hardware details in order to achieve good performance, while the emergence of HPC-focused GPUs from multiple hardware vendors, each with their own programming languages and APIs, has complicated the programming landscape. This means that porting to a new platform or GPU vendor can require substantial code rewrites, dramatically increasing the costs of software maintenance.
In this talk, I will provide an overview of some recent attempts to overcome this issue in the C++ programming space. I will discuss the current state of vendor-specific programming models, followed by an overview of some promising frameworks for vendor-agnostic development of GPU-accelerated scientific codes in high performance C++. I will compare the ease of use, support for tooling and performance of the Kokkos, SYCL and OpenMP programming models and discuss lessons learned from my time developing high-performance, GPU-accelerated molecular dynamics applications. Finally, I will discuss the implications of the changing GPU landscape for the long-term sustainability of scientific software. (Slides, Video)
Speaker
Dr Emily Kahl is a research software engineer at the Australian Institute for Bioengineering and Nanotechnology (AIBN) at the University of Queensland. Emily develops and maintains software for molecular simulation, with a specific focus on GPU-accelerated computing and machine learning methods in quantum chemistry. She is also an advocate for open-source software in computational science, and code she has developed for atomic and molecular physics has seen widespread use by Australian and international researchers.
Talk 23 - 26 October, 2.00pm AEDT - Galaxy - Linking User Needs to Distributed Computing Through the Total Perspective Vortex
The Galaxy Project and the national Galaxy Australia service make performing data driven analytics easy, by providing a simple GUI interface to users. Behind the scenes, the Galaxy Australia compute infrastructure is geographically distributed across a number of national cloud services. A user’s choice of tool and data is passed through a decision tree to ensure that the job is appropriately resourced and balances the needs of job turnaround time, throughput and overall efficiency. This is the work of the Total Perspective Vortex (TPV), a Galaxy plugin which provides a domain-agnostic design that is adaptable to other complex resource management systems. (Slides, Video)
Speaker
Gareth Price is Product Owner of Galaxy Australia, a life science tools and data analysis service (https://usegalaxy.org.au). Gareth has 20+ years’ experience as a Bioinformatician and Genomics Scientist. His expertise spans from early printed microarrays, to cartridge based GeneChips through to multiple Next Gen platforms. His work also involved a variety of model organisms from microorganisms, fruit flies, mice to humans as well as non-model organisms with limited genome information.
Dr. Nuwan Goonasekera is a research fellow at the University of Melbourne and a developer for Galaxy Australia, a life science tools and data analysis service (https://usegalaxy.org.au). Nuwan has worked on applying cloud computing technologies to the biomedical research domain since the inception of the ARDC NeCTAR Research Cloud, and has worked extensively on cloud portability and dynamically scalable infrastructure on the cloud.
Talk 22 - 24 August, 2.00pm AEST - Knowledge Graphs and Property Graphs
In this Tech Talk, Dr Car will provide a summary of NoSQL and explain the concepts of Graphs, Property Graphs and Labelled Property Graphs. He will then explain and relate the concept of Knowledge Graphs. Example technologies that implement these concepts will be covered, including Neo4J, ArcGIS Pro Knowledge, and Apache Jena Fuseki. He will conclude with some lessons learned from working in this space, and provide some key advice for those looking to explore Property Graphs and Knowledge Graphs. (Slides, Video)
Speaker
Dr Nicholas Car - Data standards and Semantic Web specialist, KurrawongAI and Adjunct at the ANU.
Dr Nicholas Car is a data standards and Semantic Web specialist working in a small Australia tech company and also as an Adjunct at the ANU. He worked as a researcher at CSIRO for almost 15 years as well as a Data Architect at Geoscience Australia, managing some of their data holdings and aiming at multi-system interoperability. He contributes to W3C and OGC standards and currently co-chairs the Australian Government Linked Data Working Group.
Talk 21 - 29 June 23, 2.00pm AEST - The Next Generation of Scientific Computing at the Australian Synchrotron
The Australian Synchrotron is a division within ANSTO and one of Australia’s premier research facilities. It produces powerful beams of light that are used to conduct research in many important areas including health and medical, food, environment, biotechnology, nanotechnology, energy, mining, agriculture, advanced materials, and cultural heritage.
After 15 years of uninterrupted operation with the original 10 experimental end stations, called beamlines, the Australian Synchrotron is currently entering an exciting new phase with the addition of 8 new beamlines. This created an opportunity for the Scientific Computing team to redesign the whole software stack from the ground up.
This presentation will take you on a journey of Scientific Computing at the Australian Synchrotron. You will learn how we employ modern, industry standard tools and architectures in a research environment in order to handle the large data throughput of modern detectors and provide the robustness our users expect from us. A particular focus will be on our use of cloud technologies, running on-premises, across our whole stack from hardware control to data processing on GPUs. (video, slides)
Speaker
Dr Andreas Moll, Manager - Scientific Computing, The Australian Synchrotron.
Andreas' computer journey started back in the day with game classics such as "Sam & Max Hit the Road" and "Command & Conquer". From these games Andreas quickly branched out into other fields such as programming, computer music and 3D modelling. Over the following years he focused his interests on software engineering, which came in handy during his studies of physics. Andreas completed his studies with a PhD in experimental particle physics during which he developed a new data processing framework for the Belle II detector in Japan. Andreas has been working at the Australian Synchrotron in various roles for the past 10 years, developing scientific software and infrastructure. Since 2017 he is the manager of the Scientific Computing team at the Australian Synchrotron.
Talk 20 - 26 April 23, 2.00pm AEST - Zero to Dev: How LAMMPS Gets Academics Coding in C++
LAMMPS is a molecular dynamics software package which, uniquely, has large chunks of contributed code written (often obviously) by academics who aren't necessarily top-notch coders (like myself!), and that's what I would explore in the talk. LAMMPS has set up a strong internal framework of classes that help academics start writing or modifying code with relatively little C++ knowledge. I can also talk about a package I recently developed with international collaborators which has been accepted into the source code and discuss some of the challenges and lessons from collaborating long-distance over coding and getting recognition for scientific coding (and the consequences when scientists aren't recognised). (Slides 1)
Speaker
Shern Tee, Postdoc, The Australian Institute of Bioengineering and Nanotechnology, University of Queensland
Contact: s.tee -at- uq.edu.au - Shern Tee is a postdoc at the Australian Institute of Bioengineering and Nanotechnology, University of Queensland, where he works on constant potential methods for molecular dynamics simulations of electrode-electrolyte nano-interfaces. Shern is also an excited advocate for research software engineering as a vital role in modern science -- whenever he's not busy with church, funky jazz, or two young and sleepless children.
Talk 19 - 23 Feb 23, 2.00pm AEDT - Data in the Time of Covid - Sources, Linkage, and Pipelines at the Victorian Department of Health
Who would have predicted that a global pandemic would supercharge health data analysis? All the technical factors necessary for a flourishing of data ecosystems were present in 2019, but they were mired in conventional ways of working and institutional inertia. Covid tore through those barriers, leading to the current data landscape, barely recognizable from the time before. This talk will describe that transition at the Victorian Department of Health, focusing on the use of data pipelines using R and the targets package (Slides 1)
Speaker
Dennis Wollersheim, Principal Analyst at DHHS Victoria.
Contact: dewoller -at- gmail.com - Dennis Wollersheim loves data. Since his first degree in computer science in 1981, where he refused to learn COBOL because it was such a travesty, he has seen all sorts, and it is just getting better. Coming over to the Victorian Department of Health as a Principal Analyst, 9 months into the pandemic, was the best thing ever. Prior to that, he taught La Trobe University health information management students to analyse million record datasets. Now, instead of expressing his joy through students, he gets to work with those datasets and more, on a daily basis.
Talk 18 - 22 Nov 22, 1.00pm AEDT - Containers for Scientific Software Distribution Service including security scanning and CVMFS
Installing scientific software on HPC often requires substantial amounts of work. In this Tech Talk we would like to get input from the community on our idea for a community-led system that makes building and distributing scientific software more sustainable. Our goal is to collect container recipes in a GitHub repository, containers are then automatically built and tested and pushed to a container registry. The containers are security rated and HPC admins can select which container subset they would like to offer to their users. The software containers will be available via a national CVMFS server infrastructure. (Slides 1, Slides 2 | Video 1, Video 2)
Speaker
Steffen Bollman, Research Fellow, University of Queensland.
Contact: s.bollmann -at - uq.edu.au - After obtaining a Master degree in Biomedical Engineering at the Ilmenau University of Technology, Steffen completed a PhD on multimodal imaging at the University Children’s Hospital and ETH Zurich, Switzerland. Dr Bollmann then joined the Centre for Advanced Imaging at the University of Queensland as a National Imaging Facility Fellow, where he pioneered the application of deep learning methods for quantitative susceptibility mapping, in the group of Prof Markus Barth. In 2019 Steffen joined the Siemens Healthineers collaborations team at the MGH Martinos Center in Boston during a 1 year industry exchange where he worked on the translation of deep learning reconstruction techniques into clinical applications. Since joining the School of Information Technology and Electrical Engineering at the University of Queensland in 2020 Dr Bollmann develops computational methods to process quantitative magnetic resonance imaging data. As part of the ARDC platform grant AEDAPT, Steffen is developing the NeuroDesk platform - a flexible, scalable, and browser-based data analysis environment for reproducible neuroimaging..Audrey Stott, Bioinformatics Platform Specialist, Pawsey
Contact: audrey.stott -at- pawsey.org.au - Audrey comes from a mixed background of podiatric medicine, wet lab and bioinformatics. From working bedside, to bench, and DevOps, her experience in these fields have led her to the area of increasing accessibility for bioinformaticians in a data-driven world. She works at the Pawsey Supercomputing Research Centre as a systems administrator to provide development and support for improving services and user experience for researchers.
Talk 17 - 11 Oct 2022, 12.00noon AEDT - TileDB Cloud; Bringing data, scientists and users together and enabling computational accessibility
Software as a Service is a practical and affordable option for executing data science projects. National Seabed Mapping's experience with TileDB has streamlined project resourcing hurdles, allowing the team to achieve large-scale processing and scientific computing outcomes helping us to drive change within the marine community.
More: https://tiledb.com/cloud/ (Slides | Video)
Speaker
Josh Sixsmith, Technical Lead, National Seabed Mapping, Geoscience Australia
Most of Josh's work at GA has revolved around QA/QC, algorithmic and data pipeline development for processing the petabytes of satellite imagery within GA's collections.
Contact: Joshua.Sixsmith -at- ga.gov.au
Talk 16 - 25 August 22, 2.00pm AEST - Making AuScope's Portal Play FAIR
In this talk I will describe the steps being taken to improve the conformance of the datasets displayed on AuScope Portal to the FAIR Data Principles (Findable, Accessible, Interoperable, Reusable). This spans a range of actions including:
(1) augmenting the display of metadata on the Portal, (2) upgrading geonetwork records to a newer standard and (3) maintaining a high standard of metadata in a geonetwork catalogue while minimising the maintenance burden. (Slides | Video)
Speaker
Vincent Fazio, Senior Engineer, CSIRO Minerals.
Contact: vincent.fazio -at- csiro.au - Vincent Fazio is a software developer with a broad range of interests including: implementing metadata standards, geospatial information systems, displaying 3D geospatial datasets, website development and open source software.
Talk 15 - 6 July 2022, 11.00am AEST - ARDC's Interactive Analytics project (inc Jupyter)
ARDC's Interactive Analytics project was created to provide Virtual Desktop and JupyterHub services to Australian researchers. The service will give researchers easier access to higher-level tools without the complexity of deploying it themselves. This tech talk will give a demo of the Virtual Desktop service, and briefly discuss the progress of the JupyterHub service. (Slides | Chat | Video below)
Speaker
Andy Botting, Systems Engineer, ARDC
Andy is Technical Lead in the Core Services team working on ARDC's Nectar Research Cloud.
Contact: andy.botting -at- ardc.edu.au
Talk 14 - 26 May 2022, 12.30pm AEST - Implementing the OHDSI OMOP Common Data Model for Medical Research
The Common Data Model allows data consolidation of Electronic Medical Records (EMR) from diverse hospital and GP sources, enhancing research breadth. The Observational Health Data Science and Informatics, Observational Medical Outcomes Partnership common data model (OHDSI OMOP or OMOP) has a very large international community working towards open source research tools and capabilities. The ARDC is investing to accelerate research in the medical domain through the conversion of key hospital and general practice EMR datasets to this model, through an EMR to OMOP public sector bridges project. In this talk, we shall present the rationale for converting data to this model and the challenges and lessons learnt in converting datasets to this model. Project DOI:10.47486/PS014 | (Slides | Video below)
Speaker
Prof Dougie Boyle, Academic Specialist in Clinical Data Analytics, University of Melbourne
Prof Boyle is a founding lead of the Australian Health Research Alliance, Transformational Data Collaboration (AHRA TDC) working towards health data quality, terminology standardisation and common data model implementation for Australia. Prof Boyle is leading an ARDC project to advance convergence of the OHDSI OMOP common data model.
Contact dboyle -at- unimelb.edu.au
Talk 13 - 30 March 2022 - Neurodesk and Neurodesktop: Software containers and CVMFS for low-maintenance, application-rich and FAIR desktop environment on PC, cloud or HPC.
NeuroDesk is a container-based yet user-friendly general scientific platform (currently tailored for the neuroscience community) that can accommodate any combination of publicly available and custom Linux applications, each packaged in its own Singularity container and distributed via CVMFS. To be able to run Neurodesk directly on a user PC, we built a lightweight Docker container, Neurodesktop, that incorporates a complete desktop, can be easily installed, and uses a standard web browser for its front end. Other than facilitating interactive analysis, Neurodesk and Neurodesktop together provide a very low entry barrier solution that enables users to build, use and share analysis pipelines without needing to master container development, and to migrate the pipelines to HPC with minimal effort. General questions to:https://neurodesk.github.io (Slides and video)
Speakers
Oren Civier, Informatics Fellow, Swinburne Neuroimaging.
Contact: ocivier -at- swin.edu.au - Dr Civier is a Linux sys admin turned researcher. His goal is to leverage cutting-edge ICT practices for the advancement of electrophysiology and science in general.Steffen Bollmann, Research Fellow, University of Qld
Contact: s.bollmann -at- uq.edu.au - Dr Bollmann develops computational methods to process magnetic resonance imaging data in a reproducible fashion.Aswin Narayanan, Informatics Fellow, National Imaging Facility Qld
Contact: a.narayanan -at- uq.edu.au - Aswin is working with imaging research data and processing infrastructure.
Talk 12 - 3 March 2022 - HPC Workflow Orchestration on Apache Airflow.
Airflow is a workflow orchestration tool that allows us to develop, schedule and run our numerical models. In this presentation I will present on how we use Airflow to conduct our Science, and some of the lessons we have learnt throughout our Airflow journey. (Slides and video).
Speaker
Blake Seers, Scientific Programmer, CSIRO Oceans and Atmosphere.
Contact: blake.seers -at- csiro.au - Blake is a Scientific Programmer in the Sea level, Waves, and Coastal Extremes team at CSIRO. He works closely with Scientists running large numerical models and works to provide HPC solutions to scientific endeavours.
Talk 11 - 22 Feb 2022 - Open Source Library Vignettes
It is a truism that Open Source software libraries come in many different flavours. In our AuScope projects we have accumulated a diverse collection of open source software library projects and we would like to share with you our experiences in creating and managing them. We will cover topics such as maintenance issues, suitability of software platforms, code separation and managing different kinds of user groups. (Slides and video).
Speaker
Vincent Fazio, Senior Engineer, CSIRO Minerals.
Contact: vincent.fazio -at- csiro.au - Vincent Fazio is a software developer with a broad range of interests including: web development frameworks, WebGL libraries and displaying geospatial datasets.
Talk 10 - 9 November 2021 - DReSA: challenges of adapting a web app to the Australasian context
This talks will discuss challenges encountered in building DReSA, which included - learning another language and web application framework (Ruby on Rails), abstracting the UI text, localisation and i18n, connecting to AAF and Tuakiri, deploying an open-source development workflow, and the biggest challenge that we encountered. (Slides | Video )
Speaker
Nick May, Software Engineeer, Pawsey, CSIRO
Nick is an accredited software engineer with over thirty years of Information Technology experience, across a variety of roles, languages, systems and domains. Following thirteen years in the research domain, in a software architecture research group, helping researchers with software and data projects and technology. More recently in the Research Software Engineering (RSE) community, as member of the organising committee for the 2019 UK RSE conference, and secretary of the RSE Association of Australian and New Zealand (rse-aunz.org). Contact: nick.may [at] csiro.au.Slides and video - see below.
Talk 9 - 25 October 2021 - CILogon: Integrated Identity and Access Management Platform
"CILogon enables researchers to log on to cyberinfrastructure (CI) by providing an integrated open source identity and access management platform for research collaborations, combining federated identity management (Shibboleth, InCommon) with collaborative organization management (COmanage). Federated identity management enables researchers to use their home organization identities to access research applications, rather than requiring yet another username and password to log on. Collaborative organization management enables research projects to define user groups for authorization to platforms."
Speakers
AU Support - AAF - Patrick Carnuccio, Solutions Analyst
John Scullen, AAF, Head Projects & Managed Services
Contact patrick.carnuccio -at- aaf.edu.au
This is a pre-recorded talk by CILogon.
Talk 9 - 25 October 2021 - CILogon: Integrated Identity and Access Management Platform
"CILogon enables researchers to log on to cyberinfrastructure (CI) by providing an integrated open source identity and access management platform for research collaborations, combining federated identity management (Shibboleth, InCommon) with collaborative organization management (COmanage). Federated identity management enables researchers to use their home organization identities to access research applications, rather than requiring yet another username and password to log on. Collaborative organization management enables research projects to define user groups for authorization to platforms."
Speakers
AU Support - AAF - Patrick Carnuccio, Solutions Analyst
John Scullen, AAF, Head Projects & Managed Services
Contact patrick.carnuccio -at- aaf.edu.au
This is a pre-recorded talk by CILogon.
Talk 8 - 3 September 2021 - Scalable Data Processing on Nectar using Apache Airflow
In this webinar, we demonstrate how to create distributed workers and micro services on Nectar using Apache Airflow technology. We have used Airflow to enable Data CO-OP capability in analysis and synthesising big datasets. This work is funded by ARC Linkage Infrastructure, Equipment and Facilities grant for the project "Data Co-operative Platform for Social Impact and Wellbeing", an infrastructure project across Swinburne, Griffith, ANU, Melbourne and UTAS.
Speakers
Amir Aryani, Soda Lab, Swinburne. Amir Aryani is the Head of the Social Data Analytics (SoDA) Lab in the Social Innovation Research Institute. The Lab applies contemporary and emerging co-op data analytics techniques to provide insight into health and social problems.
Jan Hettenhausen, Griffith. Jan is the tech Lead and software developer in the eResearch & Specialised Platforms team at Griffith University. In his 8 years in the eResearch team he’s worked on a range of bespoke research application and national platforms, is a vocal advocate of continuous delivery and DevOps and has authored and co-authored a number of conference and journal papers.
Duong Nhu, Soda Lab, Swinburne. Duong is a Research Assistant at Soda Lab, Swinburne. He is also doing a Ph.D. at Monash University, specializing in machine learning. Before Soda Lab, Duong worked as a data scientist/data engineer at various startups and IT consultancies.
Video of Talk 8 (03.09.21) - Amir Aryani, Jan Hettenhausen, Duong Nhu, Swinburne SoDA Lab, Griffith.
Talk 7 - 26 August 2021 - Container Orchestration Experiences
Abstract: Setting up a High-Availability Kubernetes cluster from scratch is “hard”, hence the popularity of public cloud providers, that do all the hard lifting for you. In this talk, I will present our AusSRC experiences setting up production-grade Kubernetes on several private and academic cloud facilities, including pitfalls and learning curves and what we would do differently.
Gordon German, Software Research Engineer, CSIRO AusSRC - Square Kilometer Array (SKA) Regional Centre Australia
Dr Gordon German is a software engineer at CSIRO Space & Astronomy, working on AusSRC software pipelines for radio astronomy data post-processsing. He has 20+ years experience in both Linux and Windows HPC systems and is currently looking at container technologies for AusSRC. (Slides)
Video of Talk 7 (17.06.21) - Gordon German, Software Research Engineer, CSIRO AusSRC.
Talk 6 - 17 June 2021 - CI/CD - Continuous Integration/Continuous Deployment
CI (Continuous Integration) and CD (Continuous Deployment) offer software development teams enhanced services for making code changes more easily and reliably. In Tech Talk 6, we hear how two teams do their flavour of CI / CD. See Slides below.
Speakers
James Tocknell, Software Developer/Data Scientist, Macquarie AAO. Australian Astronomical Observatory (AAO) is building the FAIMS software, and has extensive experience with data services/platforms for astronomy.
James Tocknell works within the Research Data and Software (RDS) team at the AAO (Australian Astronomical Optics), Macquarie University. The RDS team provides the Data Central science platform for astronomers and research teams, software development for cutting edge optical astronomical instrumentation, and is developing the latest version of the FAIMS data acquisition app. When not finishing up his PhD on outflows from protoplanetary discs, James maintains a number of open source projects, including the h5py package.
James will cover how the AAO (as a software team), h5py (as an open source project), and I (as an individual and researcher) use CI/CD, and compare and contrast the use of self-hosted vs commercial CI/CD hosted services.
James Dempsey, Senior Software Developer, CASDA CSIRO. CASDA is the data archive and services for the CSIRO ASKAP project - the pathfinder for the Square Kilometre Array project, "the world's largest and most powerful radio telescope".
James works in CSIRO Information Services Application team. He specialises in scientific data management, particularly astronomy data, and led the development of CSIRO's Data Access Portal (DAP) and CSIRO ASKAP Science Data Archive (CASDA).
Successfully releasing software is hard to do. So, the approach CSIRO take is to automate it and do it often. This is where continuous integration and continuous delivery come in. I’ll walk you through what CSIRO do with DAP and CASDA to keep our releases easy.
Slides of Talk 6
Video of Talk 6 (17.06.21) - James Tocknell, Software Developer/Data Scientist, Macquarie AAO
Video of Talk 6 (17.06.21) - James Dempsey, Senior Software Developer, CASDA, CSIRO
Talk 5 - 13 May 2021 (10.00am AEST) - Open Source Web GL Frameworks (Data Science Week)
The good, the bad and the ugly of open source 3D WebGL frameworks for data-driven geospatial applications
Abstract: The days of displaying just points on a pretty map in a web browser are fading fast. These days many web frameworks have interesting new capabilities. This includes the ability to display underground features, point cloud data and vector graphics all in 3D. In this talk I will outline the capabilities of the current crop of open source frameworks and explore their strengths and weaknesses when used to display geospatial datasets in 3D.
Vincent Fazio, Senior Software Engineer, CSIRO Mineral Resources - see slides below. VIDEO
Contact: vincent.fazio@csiro.au - Vincent Fazio is a software developer with a broad range of interests including: web development frameworks, WebGL libraries and displaying geospatial datasets.
Slides of Talk 5
Video of Talk 5 - 13 May 2021 - Vincent Fazio - Web GL
Talk 4 - 15 Apr 2021 - Platforms for Sensitive Data Analysis - Slides | Chat | Video now available and see below.
The purpose of this workshop is for developers to share experiences and learn from each other the particular problems, solutions, technical challenges and architectural decisions that have been made in their projects in order to address the unique needs of developing this type of platform. Slides | Chat | Videos now available and see below plus Intro | Discussion.
Platforms: Speakers - Org: (see 8 Bios below Platform Descriptions)
SeRP: Anitha Kannan and Jerico Revote - Monash - The Secure eResearch Platform (SeRP) software is an application suite developed by Swansea University to underpin health data research in the UK. It delivers a secure, trusted and scalable environment for data governance, linkage, control and management services for data custodians, and secure remote data analysis environments for research users. Monash SeRP, the Monash University deployment of SeRP, enables the automation of research data governance at scale. Monash data custodians can manage governance over their research data with full control and visibility over who accesses the data, where/when the data is used and how it is being used. Approved researchers, including Monash, national and international collaborators, who access data in SeRP have pre-configured tools that support SAS, SPSS, STATA, R, Python, Machine Learning/Natural Language Processing etc. (VIDEO)
AIS: Chris Albone, Fang Xu, and Ryan Sullivan - Uni Sydney - The Australian Imaging Service (AIS) is a national federation for secure imaging data management and analysis. It consists of a number of core nodes with auditable access controls and a planned fleet of edge devices for standardized de-identification and encryption of data before leaving the clinical site. (VIDEO)
SDE: Amr Hassan - Monash - Secure Data Enclaves (SDE) is a software-defined, secure, and centralised private cloud infrastructure that aims to give Monash researchers a safe environment to host, process, and analyse their sensitive data. From inception, the design was focused on creating a capability with security at its core, that did not require the user to sacrifice their overall experience. The platform architecture and design enable the team to offer each workload its dedicated enclave with well defined and monitored traffic routes to ensure that only authorised access is allowed. (VIDEO)
AusCAT: Matthew Field - UNSW. (VIDEO)
ERICA: Tim Churches - UNSW. ERICA (E-Research Institutional Cloud Architecture) is a framework which allows institutions and organisations to rapidly establish public-cloud-hosted, secure, remote access data analysis facilities for sensitive data entirely under their own control and governance, while sharing software development, maintenance and security accreditation costs with other ERICA sites. Currently there are three ERICA sites in operation, with three more to be established during 2021. This talk will cover the motivations for, capabilities and technological underpinnings of the ERICA framework. (VIDEO)
AARNet: Frankie Stevens and Robert Pocklington. The AARNet Sensitive Data service will provide researchers across the Australian university sector with a rich collaboration and analysis environment coupled with the requisite controls needed for working with sensitive data. Workflows and audit trails will enable controlled collaboration across institutions, and compliance with legislation and funding body requirements. All stages of the research lifecycle will be supported through the provision of tools for the analysis, management and storage of sensitive research data. The service will enable cross-disciplinary advances, innovation and faster research results to bring economic, commercial, environmental, social or cultural benefits to the Australian community. AARNet is a not-for-profit company that provides ultra high-speed Internet and communications services exclusively to Australia's research and education sector. (VIDEO)
OMOP PSB: Dougie Boyle and Roger Ward - Uni Melbourne (VIDEO)
AIS - Ryan Sullivan (U Sydney)
Ryan is the product manager for characterization research technology at the University of Sydney and head of the Australian Imaging Service. He has a background in microscopy and neural implant fabrication.
AIS - Fang Xu (U Sydney)
Fang Xu is the senior Devops engineer who co-designs, builds and supports Sydney University's research IT infrastructure including XNAT, CTP and Omero. Fang has a background in software development and system administration in both on-premise and cloud environments.
AIS - Chris Albone (U Sydney)
Christopher Albone is an enterprise architect at the University of Sydney. His areas of focus include research administration and research technology. He has long been involved in the implementation of XNAT and related systems at the University
SeRP - Jerico Revote (Monash)
Jerico is the lead Research Devops Systems Engineer at Monash eResearch Centre, Monash University. He has led the technical deployment of Monash SeRP in partnership with Swansea University. Jerico is also leading the technical devops for the ARDC Secure eResearch Platform project.
SeRP - Anitha Kannan (Monash)
Anitha is the Director, Research Platform Data Strategy in the Office of the Senior Vice-Provost and Vice-Provost (Research) at Monash University.
She is leading the ARDC Secure eResearch Platform project that will deploy and run the SeRP software stack as a managed, nationally consistent service via deployments and multi-tenancy arrangements at Monash eResearch Centre, Monash University.
SDE - Amr Hassan (Monash)
Amr Hassan is the Delivery leader for Technology Services and eResearch at Monash University. He leads the infrastructure platforms team at eSolutions. He holds an interdisciplinary PhD in Computational Sciences, an M.Sc in Scientific Computing, and a B.Sc. in Computer Science.
ERICA - Tim Churches (UNSW)
Tim Churches is a medically-trained epidemiologist and health data scientist, with nearly three decades of experience in designing and implementing large-scale public health and clinical research data systems, including the SURE secure data analysis facility at the Sax Institute. He is the co-founder of the ERICA project.
AARNet - Rob Pocklington
Rob is Senior Developer for AARNet's sensitive data platform.
AARNet - Frankie Stevens
Frankie is AARNet Associate Director, eResearch. She has 20 years experience in higher education / research, and is a former cancer researcher.
Talk 3 - 25 Feb 2021 - Virtual Desktop Infrastructure
Unimelb Researcher Desktops
The Researcher Desktop was designed to alleviate the burden of creating and managing virtual machines in the Melbourne Research Cloud (MRC). Researchers access the service with their existing credentials and the desktops come preinstalled with the most common research tools and the users’ OneDrive premounted. By using the same image installed on the University’s managed computers to create the Researcher Desktops, users are presented with a familiar interface and the desktops can be supported through the Service Centre. By removing the complexity associated with the IaaS approach, practically no training is required for researchers to start using the service.
Bernard Meade, Head of Research Applications and Service Integration, Research Computing Services , University of Melbourne - Bernard and his team develops and supports services that improve accessibility, ease of use and management of research focussed IT platforms provided by the Research Computing Services group.
Virtual Desktop Infrastructure at MASSIVE.
This talk will discuss the advantages, limitations and architecture of building serverless web applications in the research domain. Further, this talk will explore 2 existing research web applications Data Mosaic and Geophysical Processing Toolkit, how they take advantage of serverless technologies and the reasoning behind the architecture. Subscribe to get access to the Zoom meeting.
Jafar Lie, Research Devops Systems Engineer, Monash eResearch Centre - Jafar is an HPC DevOps Engineer, powered mainly by caffeine. He can often be found on the #HPC slack while working on containerising stuff.
Chris Hines, Senior HPC Consultant, Monash - Chris is an itinerant sys-admin, programmer and HPC consultant, with long experience in the Australian eResearch sector, who uses his skills wherever the needs of Monash University research require. His physics background means he solves problems reductively, approximating cows as spheres, to simplify the maths.
Slides - Unimelb | Monash and see videos below. Intro Video (2 mins)
20/Talk 3: Uni Melbourne Researcher Desktops: Bernard Meade
20/Talk 3: Monash Desktop Infrastructure at MASSIVE Speakers: Chris Hines, Jafar Lie
Talk 2 - 3 Dec 2020
Building Serverless Web Applications for Researchers.
This talk will discuss the advantages, limitations and architecture of building serverless web applications in the research domain. Further, this talk will explore 2 existing research web applications Data Mosaic and Geophysical Processing Toolkit, how they take advantage of serverless technologies and the reasoning behind the architecture. Subscribe to get access to the Zoom meeting.
Sam Bradley CSIRO - A software engineer and full stack developer working in the CSIRO Mineral Resources Business Unit for the last 2 years. Sam has 12 years of experience delivering all types of software projects from production web applications to experimental scientific prototypes. Slides.
Talk 20/1
8 Oct Tech Talk
New Infrastructure and Services for the Nectar Research Cloud.
Slides now available (below). Video available.
This talk will provide details of new projects to enhance the ARDC's Nectar Research Cloud including an infrastructure refresh, GPU and high memory servers to support ARDC Platforms projects, integrating capacity from commercial clouds, container orchestration services, and an interactive data analytics platform.
More: This presentation will provide an overview and some technical details of new projects to expand the ARDC's Nectar Research Cloud infrastructure and services in 2020/21. A major refresh of the Nectar Research Cloud infrastructure is underway that is expected to be completed by the end of 2020. Additional infrastructure investment in 2021 will prioritise the requirements of the ARDC Platforms projects and the provision of high-end cloud infrastructure including GPUs and high memory servers. Other projects include the development of container orchestration services, exploring integration with commercial cloud, and developing an interactive data analytics platform to provide easier access to tools such as Jupyter and R Studio.
Paul Coddington, Sam Morrison, Andy Botting, Kieran Spear, Jake Yip - ARDC Core Services team.
Past Meetings
Tech Talk previously ran 2016 - 2019 - see details at Meetup.
Talk 9 - 25 October 2021 - CILogon: Integrated Identity and Access Management Platform
"CILogon enables researchers to log on to cyberinfrastructure (CI) by providing an integrated open source identity and access management platform for research collaborations, combining federated identity management (Shibboleth, InCommon) with collaborative organization management (COmanage). Federated identity management enables researchers to use their home organization identities to access research applications, rather than requiring yet another username and password to log on. Collaborative organization management enables research projects to define user groups for authorization to platforms."
Speakers
AU Support - AAF - Patrick Carnuccio, Solutions Analyst
John Scullen, AAF, Head Projects & Managed Services
Contact patrick.carnuccio -at- aaf.edu.au
This is a pre-recorded talk by CILogon.