I'm in over my head, so this is a peter principle event, but want to give it a shot before suggesting the VP fire me and hire someone competent.
The team works on a variety of reporting and analysis projects, and everyone uses their own preferred tools (SAS, SQL, Python, Excel, Access).
The team was formed by taking "data people" out of many departments and moving to a centralized "analytics" team. The departments are now our customers.
The company also purchased a Qlikview server, licenses, and dedicated two people on the team to maintaining the environment and developing dashboards.
I have free reign to organize the team as I see fit. I'd like to move to working in a common toolset, using a project management tool, having standards/documentation, using source control, etc.
I've spent most of my career in a similar role to the other analysts on my team so don't have a ton of experience in using software development methods like this myself. I know enough to think we could benefit by them. I don't want to introduce unnecessary bureaucracy, but I am willing to add ramp up time to get people working better in the long run.
All of our data is stored in a poorly organized oracle warehouse maintained by a slow moving IT team.
The analysts independently do a lot of transformation and cleanup of data from the warehouse before being able to use it in their work.
I hope to build a pipeline where we treat the IT warehouse as our source, write an automated ETL process to clean, organize and enrich data before loading into our own database that would serve as the source for our qlikview dashboards, and other reports.
Wondering if you all could offer some advice for pulling this off.
What's the best way to introduce some standards and process in a project like this?
Possible? Or hopeless and I should commit seppuku?
Thanks!
