9 Best Articles in 2022
Towards Data Science
Build pipelines with Pandas using “pdpipe”
Towards Data Science
10 min read · · We show how to build intuitive and useful pipelines with Pandas DataFrame using a wonderful little library called pdpipe. · Shared by 47, including Dr. Ganapathi Pulipaka 🇺🇸, Carla Gentry, Linus Basig
Towards Data Science
Four Software Engineering Best Practices to Improve Your Data Pipelines
Towards Data Science
~13 min read · · From agile to abstraction, thinking about data the way we think about software can save us lots of grief. · Shared by 145
Machine Learning Mastery
Automated Machine Learning (AutoML) Libraries for Python
Machine Learning Mastery
~14 min read · · AutoML provides tools to automatically discover good machine learning model pipelines for a dataset with very little user intervention. It is ideal for domain experts new to machine learning or… · Shared by 110, including Dr. Ganapathi Pulipaka 🇺🇸, Carla Gentry, Yves Mulkers
Towards Data Science
The ultimate guide to building maintainable Machine Learning pipelines using DVC
Towards Data Science
~19 min read · · Learn the principles for building maintainable Machine Learning pipelines using DVC · Shared by 30, including rohit
Harvard Business Review
Pipelines, Platforms, and the New Rules of Strategy
Harvard Business Review
~18 min read · · Scale now trumps differentiation. · Shared by 107, including Thomas Euler, Kevin_Indig, shishir das, Grégoire Japiot 🌻, Christian P. Stobbe, Klaus Eck, Chung Chen, Chris Messina, Bram Kanstein, Luzius Meisser, Thomas Pleil, Wilko S. Wolters 🇩🇪🇪🇺, Dominik Grolimund, William El Kaim, Scott Brinker, Stephanie A Kowalski, Dom Zuend, Twitty
VentureBeat
Microsoft open-sources SynapseML for developing AI pipelines
VentureBeat
3 min read · · Microsoft has released SynapseML, a new open source project designed to make it easier to create AI model pipelines. · Shared by 23, including Bob E. Hayes
Fast Company
Offshore wind farms could do double duty as carbon capture devices
Fast Company
2 min read · · Built together, these two technologies could reduce the energy costs of carbon capture and minimize the need for onshore pipelines. · Shared by 79, including James Gingerich, @Expeflow #WorkEasier #RPA, Theodora (Theo) Lau - 劉䂀曼 🌻
Guardian Environment
Huge expansion of oil pipelines endangering climate, says report
Guardian Environment
3 min read · · More than 24,000km of pipelines planned around world, showing ‘an almost deliberate failure to meet climate goals’ · Shared by 22, including Greta Thunberg, Damian Carrington, Svein Tveitdal
cycode.com
How We Discovered Vulnerabilities in CI/CD Pipelines of Popular Open-Source Projects
cycode.com
~15 min read · · The upsurge of supply-chain incidents in recent years highlights the importance of protecting build servers and deployment processes. In the case of GitHub Actions, GitHub has implemented many… · Shared by 68
Trending
Guardian Environment
Huge expansion of oil pipelines endangering climate, says report
Guardian Environment
3 min read · · More than 24,000km of pipelines planned around world, showing ‘an almost deliberate failure to meet climate goals’ · Shared by 22, including Greta Thunberg, Damian Carrington, Svein Tveitdal
The New York Times
Germany is investigating leaks in Nord Stream gas pipelines from Russia.
The New York Times
2 min read · · Neither of the pipelines was active, but both were filled with natural gas. The sharp drop in pressure could only be caused by a leak, authorities said. · Shared by 17, including Christophe Barraud🛢🐳, Tara Imani, AIA NCIDQ ASID
The Associated Press
Record methane leak flows from damaged Baltic Sea pipelines
The Associated Press
3 min read · · COPENHAGEN, Denmark (AP) — Methane leaking from the damaged Nord Stream pipelines is likely to be the biggest burst of the potent greenhouse gas on record, by far. The Nord Stream pipeline leaks that… · Shared by 14, including David Wallace-Wells, Wolfgang Blau, Svein Tveitdal
davidabrock
Pipelines, A Tutorial
davidabrock
9 min read · · Recently, a reader wrote, asking a lot of questions about pipelines--what are they, why are they important, how do we leverage them, and so forth. At first, I was surprised by the question, I think of… · Shared by 9, including Kelly Riggs, Mike Kunkle
More like this
github.com
spotify/luigi
github.com
· luigi - Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support bu... · Shared by 31, including Mark Needham, Ludovic Claude, Nico Müller 🇺🇦, yawniek, Alexander Seifert, Jesús Torres 👩💻, Real Python
Machine Learning Mastery
A Gentle Introduction to Machine Learning Modeling Pipelines
Machine Learning Mastery
6 min read · · Applied machine learning is typically focused on finding a single model that performs well or best on a given dataset. Effective use of the model will require appropriate preparation of the input data… · Shared by 66, including Dr. Ganapathi Pulipaka 🇺🇸, Carla Gentry, rogerverhoeven, Bob E. Hayes
research.nccgroup.com
10 real-world stories of how we’ve compromised CI/CD pipelines
research.nccgroup.com
6 min read · · Mainstream appreciation for cyberattacks targeting continuous integration and continuous delivery/continuous deployment (CI/CD) pipelines has been gaining momentum. Attackers and defenders… · Shared by 18, including Matthew Green
KDnuggets
20 Basic Linux Commands for Data Science Beginners
KDnuggets
3 min read · · Essential Linux commands to improve the data science workflow. It will give you the power to automate tasks, build pipelines, access file systems, and enhance development operations. · Shared by 62
airbyte.io
ETL Pipelines with Airflow: the Good, the Bad and the Ugly
airbyte.io
7 min read · · Many data teams use Airflow for their ETL pipelines. But is using Airflow for your ETL pipelines a good practice today? Learn about the challenges of using Airflow transfer and transformation… · Shared by 14, including Marcus Borba, Mike Tamir, PhD
The New York Times
Germany is investigating leaks in Nord Stream gas pipelines from Russia.
The New York Times
2 min read · · Neither of the pipelines was active, but both were filled with natural gas. The sharp drop in pressure could only be caused by a leak, authorities said. · Shared by 17, including Christophe Barraud🛢🐳, Tara Imani, AIA NCIDQ ASID
The Washington Post
Six maps that show the anatomy of America's vast infrastructure
The Washington Post
10 min read · · A look at the United States’s bridges, electrical grid, pipelines, railroads, airports and waterways · Shared by 49, including Mark Bao, Courtney Bolton ★, Gideonro, Thomas Power, Data Science Renee, Howard Getson, Dominic Grzbielok, hiten.eth
Visual Capitalist
Interactive Map: Crude Oil Pipelines and Refineries of the U.S. and Canada
Visual Capitalist
10 min read · · Pipelines are the primary method of transporting crude oil around the world, delivering oil and its derivative products swiftly to refineries and empowering reliant businesses. And North America is a… · Shared by 15, including Evan Kirstel the $B2B Techfluencer
github.com
quantumblacklabs/kedro
github.com
· A Python library for building robust production-ready data and analytics pipelines - quantumblacklabs/kedro · Shared by 23, including Marco Unternaehrer, Thomas Spreng, Nico Müller 🇺🇦, Data Science Renee, Dr. Ganapathi Pulipaka 🇺🇸, Linus Basig
