Category: Big Data
-
Executors Killed Unexpectedly in Synapse
Some Spark platforms will hide the Yarn cluster management from an application developer. The intention is good, and the idea is to make things “easy”. The actual outcome can be very bad, however, if problems are happening that are outside your control and outside your visibility. There are a large number of platforms that repackage… Read more
-
Synapse Spark – Brownouts and Blackouts
In the past I have complained about chronic socket exceptions on Microsoft private endpoints (PE’s and MPE’s). You can find a Christmas story about that here for example. Of course, these anecdotes from a single Synapse customer aren’t always convincing . Others might thing I’m exaggerating the problem or spinning it to suit my purposes.… Read more
-
A Christmas with Synapse Analytics
It is now 2024, and I’ve decided to write a post about Synapse Analytics in Azure. I use Spark in Synapse to prepare data for analysis – primarily for the benefit of OLAP users and client applications. I’ve been working with Spark in Synapse for a couple of years now. This platform is best described… Read more