Can i combine different tools
If you have data in various third party services like Google Analytics and several different databases, you can access all of it transparently from one place with Data Virtuality.
To connect your databases and configure the application, you use a simple graphical interface. Popular tools such as Tableau can connect to it afterwards and access data from connected data sources without breaking a leg or struggling with the amount. In the background, Data Virtuality is usually configured to load data into Amazon Redshift intelligently, which reduces the load on the source databases and speeds up queries significantly.
The software requires a regular licensing fees apply. Want to load data periodically from various databases and logs, while being able to perform long running, complex chained operations on it? You also really like Python? Luigi was developed by Spotify and is currently used by many successful companies such as Buffer or Stripe. It is widely used, has a strong community and is very developer friendly.
In case something breaks or just to see how everything is going, you can get a visual overview of your whole pipeline with no extra effort.
This clean and readable open source project is very versatile and fits well into the Python and Hadoop data-processing ecosystem. You need to write code to use it - with great power comes great responsibility. Luigi is meant for projects that crunch data in regular intervals, and is perfect for creating maintainable and elegant data processing pipelines. Described as an event analytics platform , this approach differs from the two above.
It focuses on generating Snowplow-specific events and processing them, instead of transforming data from existing databases, services or logs. The SaaS scene is huge these days, with tons of software solutions available. Figure out what you want to build — and then pick the suitable building blocks. The Trello cards we choose to push into development get made into tasks on the Teamweek timeline.
Also, on our weekly meetings we run Teamweek on the office big screen and schedule the upcoming week with simple drag and drop actions. Teamweek, then, is used for keeping an eye on the strategy and the big picture. When it comes to managing the intricate programming work, we take the Teamweek tasks and set them up as milestones in Github.
These milestones then get broken up into the various smaller issues and tasks:. Putting all the blocks together, our multi-tool workflow looks something like this:. Toggl for tracking efficiency. All together, they make for one amazing planning tool that is still simple to use. We hate complicated things. Check out how to combine Toggl time tracking with Kanban methodology.