You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2023-01-19-SQLBits.md
+2-1Lines changed: 2 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -24,12 +24,13 @@ On a sidenote: Many sessions this year evolve around database development & depl
24
24
## Training Sessions (Tuesday & Wednesday)
25
25
26
26
The [training sessions](https://events.sqlbits.com/2023/training-days) have/had to be booked in advance, so i picked those two:
27
+
27
28
-[Lakehouse in a day](https://events.sqlbits.com/2023/Lakehouse-in-a-day)
28
29
-[Power BI and Synapse: Leverage The Best of Both For Scale](https://events.sqlbits.com/2023/power-bi-and-synapse)
29
30
30
31
The latter one must sound like a joke to people who know me and my dislike for reporting/frontend topics. That dislike is due to me spending 5+ years on designing & implementing financial reports in a multitude of frontends. I really feel like i have done my part in that regard (for this lifetime and the next). However, this specific session seems to be more focused on the modeling side of things in Power BI, in accordance with various Synapse architecture approaches at different scales. This will be quite interesting as well as extremely relevant for my daily work. Besides, attending this session will allow me to be even more of a smartass who tells his colleagues on how to do their job (feeling a **bit** bad for them already...).
31
32
32
-
Regarding Lakehouse in a day: I really don't know what could possibly take a whole day when building a lakehouse (i can do it in 45 minutes tops:stuck_out_tongue_closed_eyes:). Stupid jokes aside, i am curious and very much looking forward to that session, since my practical experience in building lakehouse(s) in _real_ projects & business cases is still nowhere near the point i'd like it to be (and where i start to feel comfortable). I hope the speakers/trainers are well prepared, because i will for sure come up with some nasty questions and give them a run for their money.
33
+
Regarding Lakehouse in a day: I really don't know what could possibly take a whole day when building a lakehouse (i can do it in 45 minutes tops!). Stupid jokes aside, i am curious and very much looking forward to that session, since it will look at lakehouses from various perspectives (Databricks, Azure Data Factory, Synapse). But the real reason for attending is to finally meet Simon Whiteley in person, after having watched tons of his videos in the past. And the best thing is: he can't even run away from my (stupid) questions :stuck_out_tongue_closed_eyes:. The Synapse Espresso guys are co-hosting, so quite the "celebrity" lineup, at least in data engineering terms.
Copy file name to clipboardExpand all lines: _tabs/about.md
+32-31Lines changed: 32 additions & 31 deletions
Original file line number
Diff line number
Diff line change
@@ -15,53 +15,59 @@ I am Thomas and currently occupied as an (Azure) data engineer at [Capgemini Aus
15
15
Not very surpsingly, my main focus is on all things data. More specifically, that currently means:
16
16
17
17
- Databricks, Spark & Delta
18
-
- Synapse (Serverless & Dedicated[^dedic], Spark, Data Explorer/Kusto, ...)
18
+
- Delta Live Tables & Spark Structured Streaming
19
+
- Data Explorer/Kusto
19
20
- Fabric
20
-
- Data Pipelines (Synapse & Data Factory)
21
-
- SQL-Servers (of any kind)
22
-
- Datalakehouses
23
-
- Streaming (data, not movies...)
24
-
- Restful APIs & GraphQL
21
+
- Azure Data Factory
25
22
- Cloud Infrastructure as Code (Terraform, Bicep)
26
23
27
-
After 18 years of data warehousing, i am now pushing lakehouse architectures and the idea of a unified data platform. Besides signing the [DataOps Manifesto](https://dataopsmanifesto.org/), i am also a supporter of the "treat data as a product" movement.
24
+
And not so currently, meaning in the recent past i had a lot of:
25
+
26
+
- Synapse (Serverless & Dedicated[^dedic])
27
+
- SQL-Server Warehouses
28
+
- Restful APIs & GraphQL
29
+
30
+
After 18 years of classical data warehousing, i finally got to dive into various (data-)lakes. And once in the lake(-house), there is no turning back, that's for sure.
31
+
32
+
Besides signing the [DataOps Manifesto](https://dataopsmanifesto.org/), i am also a supporter of the "treat data as a product" movement.
28
33
29
34
## Side-Shows
30
35
31
-
Since one of my main personal treats is being lazy in combination with a deep hatred for repetitive tasks, i developed a natural interest in DevOps & automation. To make up for lack of concentration (and sleep...), i am usually not touching any code unless it's version controlled. That includes but is not limited to: cloud infrastructure (as code), database schemas, pipeline definitions, all kinds of scripts/notebooks and the grocery list for my next shopping tour.
36
+
Since one of my main treats is lazyness when it comes to repetitive tasks, i developed a natural interest in DevOps & automation. And due to constant lack of sleep, I usually don't touch any code, unless it's version controlled. That includes but is not limited to: cloud infrastructure (as code), database schemas, pipeline definitions, all kinds of scripts, notebooks and the grocery list for my next shopping tour.
37
+
38
+
As a result of the above, i somehow stumbled into the beautiful world of DevOps (although that name was not a thing back then). My personal project lifecycle these days often goes like this: Getting hired as data architect/engineer and then silently converting into the dedicated DevOps and Git guy. It was never my intention, but i think i got the DevOps Bingo card full now:
32
39
33
-
As a result of the above, a topic that attracted my interest (and since then also became part of my professional portfolio) is the implementation/optimization of development & deployment workflows in data projects. Some of the tools/services that i have been using in that regard are:
40
+
- Azure DevOps (Repos, Boards, Pipelines, etc.)
41
+
- The Atlassian(s): Bitbucket, Jira, Confluence (and even Bamboo...).
42
+
- Github (my teenage love)
43
+
- And lately also a lot of: Gitlab
44
+
- And if artefact registries also count on that list: Cloudsmith (great tool by the way)
However, there can be only one god. And for me that is and always will be: __Git__[^git]. After using it for almost a decade now, i still feel like i don't know half of it and am constantly amazed how limitless the possibilities of this genious piece of software really are.
40
47
41
-
Since __DevOps and automation__ usually require some sort of scripting, i am constantly trying to improve my Powershell, Python, Bash & Fish hacking skills.
48
+
## Scripting
42
49
43
-
Regarding __data manipulation & transformation__ i am most experienced in T-SQL, due to a strong SQL Server background. These days though, i mainly work with Python/Pyspark & Spark-SQL in (Jupyter or Databricks) notebooks, which is a lot of fun and a welcome change.
50
+
Since __DevOps and automation__ usually require some sort of scripting, i also found myself dealing with the usual (scripting) suspects: Powershell, Bash, Python (and for fun and good looks also: Fish).
44
51
45
-
My first IT related job as a web developer (end of last century...) still benefits me to date! Having basic knowledge about HTTP and OAUTH (flows) can be really helpful when using or providing REST-APIs. Not my favorite topic in general, but like in real life (relationships): some things just need time to grow :grin:
52
+
Regarding __data manipulation & transformation__ i am most experienced in T-SQL, due to spending so many years implementing SQL warehouses. And for quite a while, writing data transformations in Python felt somehow "wrong" to me. But today there is no doubt about it in my mind: The added value that you get (for free) from the whole Python software engineering ecosystemn (formatting, linting, automated testing, packaging and so many more) cannot be missed out on. Implementing data projectss almost starts to feel like real software devleopment these days.
46
53
47
-
In the past i also pretended to know a bit of: Lua, Tcl/Tk, VBA, Javascript, Actionscript and probably a few other languages which i forgot a long time ago.
54
+
And a life without notebooks might be possible, but makes abosulety no sense to me :smirk:.
48
55
49
56
## Professional Past
50
57
51
-
Some technologies, software providers and topics that i spent time with in my (professional) past are:
58
+
Way back in time, i also had to work a lot with (or in):
- IBM Planning Analytics (aka TM1), Palo/Jedox and other _real_ OLAP engines & databases including all sorts of Excel integrations that usually come along with those.
56
-
- Classical Data Warehousing & Modeling.
57
63
- Corporate Performance Management (CPM): Main focus being financial, sales, project and product planning & forecasting.
58
64
- Financial Reporting: Implementation of financial reports for all kind of flavors and in different tools & frontends as well as supportive addins (like Graphomate). Also creation of notation concepts/standards for financial reporting (IBCS-like).
59
65
- As an inhouse consultant, i also spent quite some time in improving the collaboration toolset and processes in the central Controlling department (40 people spread over various locations, countries and languages). Which basically means that i played around a lot with: Microsoft Teams (plus Addins/Apps), Sharepoint (on-premise and online), Planner/ToDo, OneDrive, O365, ...).
60
66
- At the same time i also made some operational use of my educational/business background by conducting the monthly business consolidation (as well as providing financial reports) for over 100 legal entities in [Styria Media Group](https://www.styria.com).
61
67
62
68
## Education & Certifications
63
69
64
-
My educational background is a master's degree in "Management and International Business"[^mib] (yikes) and 35+ years of being a self-taught computer nerd. If you care about certifications (personally i don't), you can check my [Credly](https://www.credly.com/users/ttotter) or [MS-Learn](https://learn.microsoft.com/en-us/users/thomastotter-5644/) pages and might find out that (besides the obvious data engineering certs) i am also not a complete stranger to agile methods.
70
+
My educational background is a master's degree in "Management and International Business"[^mib] (yikes) and 35+ years of being a self-taught computer nerd. If you care about certifications (personally, i don't), you can check my [Credly](https://credly.credly.ttotter.pw) or [MS-Learn](https://mslearn.totter.pw) pages and might find out that (besides the obvious data engineering certs) i am also not a complete stranger to agile methods.
65
71
66
72
However, my opinion towards certifications is best explained by this picture (borrowed from the great Martin Fowler[^mf]):
67
73
<br/>
@@ -70,7 +76,7 @@ However, my opinion towards certifications is best explained by this picture (bo
70
76
71
77
## Creed
72
78
73
-
Since i couldn't put my work ethics/beliefs to better words myself, i am _borrowing_ most of [Automattic's creed](https://automattic.com) here:
79
+
Since i couldn't put my work ethics/beliefs to better words, i am _borrowing_ most of [Automattic's creed](https://automattic.com) here:
[^blog]: Powered by [Github Pages](https://pages.github.com/) and [Chirpy](https://github.com/cotes2020/jekyll-theme-chirpy)
104
107
@@ -113,5 +116,3 @@ You can find, stalk, or contact me here:
113
116
[^assig]: Greetings go out at to all my former POs, project leads and bosses :wave:
114
117
115
118
[^corny]: Ok this one sounds a bit corny considering i am mainly working with Microsoft products and services. But it's the idea that counts!
116
-
117
-
[^mast]: <arel="me"href="https://mastodon.social/@brain246">Mastodon.Social</a> and <arel="me"href="https://me.dm/@brain246">Mastodon.Medium</a> forced me to place these links here :imp:
0 commit comments