My speciality is business data, more specifically, data at the edges. Data that has been extracted from, or needs to be generated outside of, formal systems. Such data tends to be the extremely valuable and useful, but only if shaped into a format that has meaning to those who can use and analyse it. I've been extracting, marshalling and shaping such data for most of my 30 years in the IT business.
The technical term for what I do is ETL (Extract, Transform & Load). It's the name given to the skill-set and tools associated with the movement of datasets. It's the unglamorous, but essential backbone, to a whole range of IT deliverables: Data Warehousing, BI (Business Intelligence), Planning & Forecasting, New System Commissioning, Systems Integration.
In the past my skills would only have been of use to large IT departments, as major data shaping activities such as BI & data warehousing were owned by IT; small businesses and departmental workgroups were outside the loop except as providers of requirements (and money) and as long-suffering "end users".
This alienation from the process of data shaping and the need to simply "get things done" led to the evolution of "shadow IT", usually involving Excel is some form or another. Although many attempts have been made to control and tame this wild-west of data creation and consumption, they've largely failed. With good reason, the best people to get value out of data are those who depend on and generate the data in the first place.
I've spent a large part of my career helping business users bridge that "final mile" where formal systems stop and business needs begin.
Career:My background in IT goes back over 30 years, starting as a trainee programmer with one of the UK's earliest software houses, Altergo. After 4 years with Altergo (the only ''proper job' I've ever had!) I spent the next decade or so freelancing as an OLTP developer, first VAX/VMS based, then Oracle solutions, working for companies such as PA Consulting, Digital (DEC), IDA Ireland, Woodchester Finance, & Guinness (now Diageo). In the early 90s I co-founded Keyhouse Computing, now the largest supplier of legal software in Ireland, while continuing to act as a consultant for Guinness Ireland (Diageo). Since selling my interest in Keyhouse, I've continued to freelance and provide consultancy in my specialities of BI and ETL with an increasing interest in final-mile solutions using Excel. My BI/ETL consulting engagements during this time have included Jedox (makers of PALO), Constellation Software (A Canadian/US niche-ERP company) and SAGE CRM. I now share my thoughts on all things BI,ETL and Excel with all and sundry via my long running blog Gobán Saor - A Country Datasmith. I'm also the developer of several free Excel-based micro ETL tools. | Business Domains:A large part of my career (15 years) was spent in the Sales & Marketing side of the FMCG sector, working on a series of CRM (Siebel), ERP (SAP & bespoke Oracle) and datawarehousing/BI projects. It was also in this industry there that I discovered the power of Excel and its effective use by civilians working in "the shadow IT" world. In the past I've worked as a consultant in the public sector (working for PA Consulting in the UK public service and as a contractor for IDA Ireland). My experience in manufacturing includes brewing (Guinness) and hi-tech (Digital). As you see mostly PLCs or large government agencies, but I also know business from a small business perspective having being either freelance or a partner in a small business (Keyhouse) for most of my IT career. So I've done the taxes, the accounts, the bill issuing and debt collecting, cash flow and sales forecasting and so on in situations where my own fortunes were at stake, not those of an employer or customer. I co-founded two SME-focused businesses, one failed (Lanson Systems) , the other, Keyhouse Computing, succeeded. Both were primarily involved in packaged accounting and work-flow solutions for the legal market but also developed bespoke solutions for Finance (Woodchester) and FMCG (Diageo) industries. | Technologies that have shaped me:From the mid 80s my focus became Oracle; working both as a developer (PL/SQL, SQL*Forms, SQR) and a DBA (data modelling, rather than sysadmin-type role). From the late 90s I began working on ETL, data warehousing, budgeting and BI projects and on ERP (SAP SD,FI,MM) and CRM (Siebel) implementations. My SAP and Siebel work revolved around data take-on, systems integration with legacy systems and with feeding and utilising Data Warehouse and BI systems. As my roles changed from developer/DBA to business/data analyst I began to depend more and more on spreadsheets, thus began my fascination with all things Excel. The mid 2000s I turned my attention to more agile technologies such as the new emerging web frameworks: Ruby-on-Rails, Django etc. It was through this work that I discovered Python and SQLite, two enormously powerful, yet essentially simple, tools. In the last few years I've also discovered and mastered data related open-source tools such as PALO (Excel and web-based OLAP tool),Talend ETL and Mondrian (an MDX speaking, ROLAP-with-a-cache tool). I've moved most of my development infrastructure to the cloud. Utilising Amazon's EC2 and S3 for data processing and storage. When MS released the PowerPivot add-in form Excel 2010 I became an early adopter, recognising it as a huge advance in 'final-mile' BI. As a virtual data analyst, I no longer have access to a lot of the expensive enterprise software I once depended on, instead I have rolled-my-own datasmithing tools, microETL, stitching together Excel,SQlite and PowerPivot with C, C#,Python,JavaScript,VB.NET and a lot of VBA. |