lingo.lol is one of the many independent Mastodon servers you can use to participate in the fediverse.
A place for linguists, philologists, and other lovers of languages.

Server stats:

65
active users

#apachespark

0 posts0 participants0 posts today
Dirk Van den Poel<p>Today’s online lecture of my <a href="https://mastodon.online/tags/BigData" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>BigData</span></a> class is on introducing <a href="https://mastodon.online/tags/PySpark" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>PySpark</span></a> for data science <a href="https://mastodon.online/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>MachineLearning</span></a> <a href="https://mastodon.online/tags/orms" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>orms</span></a> <a href="https://mastodon.online/tags/python" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>python</span></a> <a href="https://mastodon.online/tags/DataScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>DataScience</span></a> <a href="https://mastodon.online/tags/dataanalytics" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dataanalytics</span></a> <a href="https://mastodon.online/tags/ApacheSpark" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ApacheSpark</span></a> <a href="https://mastodon.online/tags/SQL" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>SQL</span></a></p>
rmoff 🏃🏻 🍺 🥓<p>✍🏻 Final part of my blog series on Write-Audit-Publish (WAP), in which I show in detail how to implement it using <a href="https://data-folks.masto.host/tags/apacheSpark" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>apacheSpark</span></a>, <span class="h-card"><a href="https://social.lfx.dev/@deltalakeoss" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>deltalakeoss</span></a></span>, <a href="https://data-folks.masto.host/tags/Minio" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Minio</span></a>, and <span class="h-card"><a href="https://data-folks.masto.host/@lakeFS" class="u-url mention" rel="nofollow noopener" target="_blank">@<span>lakeFS</span></a></span> </p><p><a href="https://lakefs.io/blog/write-audit-publish-with-lakefs/?utm_campaign=Social%20media%20activity&amp;utm_source=Mastodon&amp;utm_medium=social&amp;utm_content=blog_rm-wap3" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="ellipsis">lakefs.io/blog/write-audit-pub</span><span class="invisible">lish-with-lakefs/?utm_campaign=Social%20media%20activity&amp;utm_source=Mastodon&amp;utm_medium=social&amp;utm_content=blog_rm-wap3</span></a></p><p>---</p><p>📝part 1: 🙋🏻What is WAP? <a href="https://lakefs.io/blog/data-engineering-patterns-write-audit-publish/?utm_campaign=Social%20media%20activity&amp;utm_source=Mastodon&amp;utm_medium=social&amp;utm_content=blog_rm-wap1" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="ellipsis">lakefs.io/blog/data-engineerin</span><span class="invisible">g-patterns-write-audit-publish/?utm_campaign=Social%20media%20activity&amp;utm_source=Mastodon&amp;utm_medium=social&amp;utm_content=blog_rm-wap1</span></a></p><p>📝part 2: 🛠️ Comparing how different tools implement WAP <a href="https://lakefs.io/blog/how-to-implement-write-audit-publish/?utm_campaign=Social%20media%20activity&amp;utm_source=Mastodon&amp;utm_medium=social&amp;utm_content=blog_rm-wap2" rel="nofollow noopener" target="_blank"><span class="invisible">https://</span><span class="ellipsis">lakefs.io/blog/how-to-implemen</span><span class="invisible">t-write-audit-publish/?utm_campaign=Social%20media%20activity&amp;utm_source=Mastodon&amp;utm_medium=social&amp;utm_content=blog_rm-wap2</span></a></p><p><a href="https://data-folks.masto.host/tags/writeAuditPublish" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>writeAuditPublish</span></a> <a href="https://data-folks.masto.host/tags/dataEngineering" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>dataEngineering</span></a> <a href="https://data-folks.masto.host/tags/datadon" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>datadon</span></a> <a href="https://data-folks.masto.host/tags/opensource" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>opensource</span></a></p>
Carlos Peña<p>Yesterday we tried to upload 10M rows to an <a href="https://infosec.exchange/tags/Azure" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Azure</span></a> Table using <a href="https://infosec.exchange/tags/ApacheSpark" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>ApacheSpark</span></a> and <a href="https://infosec.exchange/tags/Databricks" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Databricks</span></a>. We hit 333K rows per minute at our best.</p><p>Wondering if anyone here has done something similar?</p><p>From what I read, the max transactions per second on an Azure Table is 20K so… I guess we can try to speed it up a bit further.</p>