lingo.lol is one of the many independent Mastodon servers you can use to participate in the fediverse.
A place for linguists, philologists, and other lovers of languages.

Server stats:

66
active users

#datapipelines

0 posts0 participants0 posts today
Kedro<p>In the pipeline: May 2024 edition! 🔶</p><p>This month we published several releases across the Kedro ecosystem, got featured in O'Reilly's new book "Software Engineering for Data Scientists", published more videos in our YouTube channel, and got accepted at several Python conferences.</p><p>Are you at <span class="h-card" translate="no"><a href="https://fosstodon.org/@pycon" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>pycon</span></a></span>? Don't miss Juliana Ferreira's talk about Kedro!</p><p>Full blog post:</p><p><a href="https://kedro.org/blog/in-the-pipeline-kedro-news-05-24" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">kedro.org/blog/in-the-pipeline</span><span class="invisible">-kedro-news-05-24</span></a></p><p><a href="https://social.lfx.dev/tags/kedro" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>kedro</span></a> <a href="https://social.lfx.dev/tags/python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>python</span></a> <a href="https://social.lfx.dev/tags/pydata" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>pydata</span></a> <a href="https://social.lfx.dev/tags/datascience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datascience</span></a> <a href="https://social.lfx.dev/tags/mlops" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>mlops</span></a> <a href="https://social.lfx.dev/tags/kedroviz" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>kedroviz</span></a> <a href="https://social.lfx.dev/tags/datapipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datapipelines</span></a></p>
Juan Luis<p>Today's random thought: thinking about <span class="h-card" translate="no"><a href="https://social.lfx.dev/@kedro" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>kedro</span></a></span> for ETL pipelines, the Singer ecosystem, Polars dataframes, Voltron's Codex, data contracts...</p><p>Is there even a standard for _data schemas_? I've seen Singer schemas, SQL `CREATE` statements, JSON Schema, and possibly more.</p><p><a href="https://social.juanlu.space/tags/data" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>data</span></a> <a href="https://social.juanlu.space/tags/etl" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>etl</span></a> <a href="https://social.juanlu.space/tags/datapipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datapipelines</span></a></p>
Kedro<p>Big announcement: our 3 hour video introduction to Kedro is live! 🔥 <a href="https://youtube.com/playlist?list=PL-JJgymPjK5LddZXbIzp9LWurkLGgB-nY&amp;si=EED0GXyBPxP5osUT" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">youtube.com/playlist?list=PL-J</span><span class="invisible">JgymPjK5LddZXbIzp9LWurkLGgB-nY&amp;si=EED0GXyBPxP5osUT</span></a></p><p>The course covers an introduction to what Kedro is, a walkthrough of the Spaceflights tutorial using both Jupyter Notebook and VSCode, new stuff in Kedro 0.18 like dataset factories and the OmegaConfigLoader, how to use S3 and MinIO for your filepaths, and how to deploy Kedro to a Docker image or Airflow.</p><p>We hope you enjoy it! 🔶</p><p><a href="https://social.lfx.dev/tags/python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>python</span></a> <a href="https://social.lfx.dev/tags/pydata" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>pydata</span></a> <a href="https://social.lfx.dev/tags/datascience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datascience</span></a> <a href="https://social.lfx.dev/tags/datapipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datapipelines</span></a> <a href="https://social.lfx.dev/tags/kedro" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>kedro</span></a> <a href="https://social.lfx.dev/tags/kedroviz" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>kedroviz</span></a></p>
Kedro<p>New blog post: How to share a Kedro-Viz pipeline visualisation 🔶</p><p>In this blog post, our colleague Nero Okwa explains how to set up and use "publish and share" to host a Kedro-Viz project on Amazon S3, one of our most requested features.</p><p><a href="https://kedro.org/blog/how-to-share-a-kedro-viz-pipeline-visualisation" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">kedro.org/blog/how-to-share-a-</span><span class="invisible">kedro-viz-pipeline-visualisation</span></a> </p><p><a href="https://social.lfx.dev/tags/kedro" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>kedro</span></a> <a href="https://social.lfx.dev/tags/python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>python</span></a> <a href="https://social.lfx.dev/tags/pydata" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>pydata</span></a> <a href="https://social.lfx.dev/tags/datascience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datascience</span></a> <a href="https://social.lfx.dev/tags/dataviz" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dataviz</span></a> <a href="https://social.lfx.dev/tags/dashboarding" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dashboarding</span></a> <a href="https://social.lfx.dev/tags/datapipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datapipelines</span></a></p>
Kedro<p>Our colleague Nero Okwa has recorded some videos on how to get started with Kedro Viz 🔶</p><p>Check them out! <a href="https://youtu.be/08MvSbw75us?si=yC7QcwlNFI5-25h8" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">youtu.be/08MvSbw75us?si=yC7Qcw</span><span class="invisible">lNFI5-25h8</span></a></p><p><a href="https://social.lfx.dev/tags/kedro" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>kedro</span></a> <a href="https://social.lfx.dev/tags/kedroviz" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>kedroviz</span></a> <a href="https://social.lfx.dev/tags/dataviz" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dataviz</span></a> <a href="https://social.lfx.dev/tags/datapipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datapipelines</span></a> <a href="https://social.lfx.dev/tags/datascience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datascience</span></a> <a href="https://social.lfx.dev/tags/python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>python</span></a> <a href="https://social.lfx.dev/tags/pydata" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>pydata</span></a></p>
CrateDB<p>Looking to streamline your data pipelines? 💡<br>In this tutorial, Marija Selakovic, CrateDB's Developer Advocate, shows you how to streamline data ingestion, transformation, and loading (ETL) processes with just a few lines of Python code 📊</p><p>Check out the tutorial here: 👉 <a href="https://hubs.ly/Q020TZs50" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="">hubs.ly/Q020TZs50</span><span class="invisible"></span></a></p><p><a href="https://fosstodon.org/tags/db" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>db</span></a> <a href="https://fosstodon.org/tags/data" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>data</span></a> <a href="https://fosstodon.org/tags/database" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>database</span></a> <a href="https://fosstodon.org/tags/CrateDB" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CrateDB</span></a> <a href="https://fosstodon.org/tags/Prefect" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Prefect</span></a> <a href="https://fosstodon.org/tags/DataPipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataPipelines</span></a> <a href="https://fosstodon.org/tags/Python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Python</span></a> <a href="https://fosstodon.org/tags/ETL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ETL</span></a></p>
Kedro<p>It's release week 🔥</p><p>🔶 kedro 0.18.12 with dataset factories<br>🔶 kedro-datasets 1.5.1 with lazy loading and `pandas.DeltaTableDataSet`<br>🔶 kedro-airflow 0.6.0 with an option to configure DAG kwargs using `airflow.yml`<br>🔶 kedro-telemetry 0.2.5 paving the way for Python 3.11 support</p><p>Thanks to our various community contributors! 💛 Check out the complete release notes in our GitHub organisation <a href="https://github.com/kedro-org/" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="">github.com/kedro-org/</span><span class="invisible"></span></a></p><p><a href="https://social.lfx.dev/tags/kedro" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>kedro</span></a> <a href="https://social.lfx.dev/tags/python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>python</span></a> <a href="https://social.lfx.dev/tags/pydata" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>pydata</span></a> <a href="https://social.lfx.dev/tags/datascience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datascience</span></a> <a href="https://social.lfx.dev/tags/datapipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datapipelines</span></a></p>
Fabian Stadler<p>Ever wanted to write <a href="https://mastodon.fabianstadler.com/tags/ETL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ETL</span></a> pipelines in bare code - with little thought 🤔 ?</p><p>With <a href="https://mastodon.fabianstadler.com/tags/DeltaLiveTables" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DeltaLiveTables</span></a>, the latest ETL framework from <a href="https://mastodon.fabianstadler.com/tags/Databricks" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Databricks</span></a>, you can leverage the power of code to create reliable, scalable, and easy-to-maintain <a href="https://mastodon.fabianstadler.com/tags/DataPipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataPipelines</span></a> 🚀 . Join me as I explore the framework and share my experiences with it.</p><p><a href="https://www.fabianstadler.com/2023/04/data-pipelines-as-code-with-delta-live-tables.html" rel="nofollow noopener" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">fabianstadler.com/2023/04/data</span><span class="invisible">-pipelines-as-code-with-delta-live-tables.html</span></a></p>
Gwyn<p>Okay, <a href="https://mastodon.social/tags/Introduction" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Introduction</span></a> time.</p><p>I'm a <a href="https://mastodon.social/tags/DataEngineer" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataEngineer</span></a> in <a href="https://mastodon.social/tags/Perth" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Perth</span></a> who keeps trying to pick up her <a href="https://mastodon.social/tags/clarinet" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>clarinet</span></a> again.</p><p>I'm also a <a href="https://mastodon.social/tags/vegetarian" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>vegetarian</span></a> which I say to preface that I am trying to <a href="https://mastodon.social/tags/LearnToCook" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>LearnToCook</span></a> from the very basics, to establish a <a href="https://mastodon.social/tags/CookingHabit" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>CookingHabit</span></a> so I'm starting *very* slowly. Like learning to chop + cook specific things.</p><p>Otherwise, I spend most of my time <a href="https://mastodon.social/tags/coding" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>coding</span></a> , building <a href="https://mastodon.social/tags/DataPipelines" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataPipelines</span></a> (yes, at home, in my free time), and <a href="https://mastodon.social/tags/renovating" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>renovating</span></a> our home, for which we paid off the mortgage just under a year ago!</p>