… I work for myself publishing websites, currently UK train times mostly , eg., traintime.uk, and have longer term plans not just for rail but public transport systems in general. Key to my plans are the ever increasing availability of open data and the efficient processing thereof. To this purpose I guess Kafka could be a useful tool towards that goal.
I do have an opening question and it is this; would Kafka be a useful tool towards that goal?
I currently develop on Windows and host on Centos 7 with Apache HTTP server. I’m currently transitioning from predominantly PHP / MySQL to Java / Postgres and have also decided on using Wildfly as my first choice application server, and Hibernate for ORM. I’ll be ready for my first live project on my new platforms in a few months and the main first task will be to populate the database with feeds of UK rail data (a couple of open data streaming feeds provided by National Rail and Network Rail). Having been kindly pointed to this forum by the host organiser of a recent meetup event following a great talk on the subject by Robin Moffat I’m hoping for at least one person to know what I’m talking about I’m looking for the easiest, simplest, most efficient, etc., (best!) way to get that data onto my database. Is Kafka the tool for the job?