It’s an honor and a privilege to have helped 2.6 million developers and earned over 25K points on Stack Overflow. For those who don’t know, Stack Overflow is a Q&A site similar to Quora and Reddit that is essential to life as a software engineer. Most of us turn to it at least once a day to see if anyone else has tackled the same problems, and far more often than not, someone has.
The 2017 Stack Overflow Developers Survey had the most respondents since they began the project in 2011. You really should take a look. They cover a lot of ground, and the findings across geography and demographics are fascinating. It’s interesting most developers report feeling underpaid. That isn’t surprising to me, but it might be counterintuitive to people more accustomed to Stark Industries than to Pied Piper. It’s natural to follow up by asking which programming languages pay the most, and those answers did surprise me.
As promised earlier this year, we at Vidya are proud to officially announce our newest course Analytics with Apache Spark. Spark is a cool technology making an enormous–and growing–impact in the Big Data space, so naturally there are a lot of courses out there. Ours is different. Naturally we spend a lot of time on Spark itself with numerous code examples and challenging exercises, but we also stress the importance of things that have always mattered and still matter–architecture, security, and software engineering concepts like unit and integration testing, continuous integration, and continuous delivery.
Vidya is proud to be working with Thomson Reuters Special Services, a leading provider of threat detection solutions. Their software analyzes billions of public and proprietary records with innovative technology to deliver realtime, actionable intelligence to support sound decision making. We have joined a team of senior engineers with a wide variety of expertise. Currently, we are using Play Framework in Scala as the web application framework with an AngularJS interface to a backend MongoDB database.
At Vidya we currently offer two courses, Software Engineering in Java and Agile Software Project Management with Scrum. In response to popular demand…OK, like eight or nine people…we are currently working on a third course to be ready by Summer 2015 tentatively called Analytics with Apache Spark. As “Big Data” becomes more and more of a thing, there just aren’t enough software engineers who know the tools and techniques for doing meaningful, performant, cloud-scale analytics.
Please take a look at my latest column for Government Computing News where I gently introduce you to the basics of Hadoop and describe cool technologies that build on it. Just to give you an idea, here is the unedited introduction. It’s a funny word. You have only a vague notion of what it is. You’ve heard that it takes a lot of work but is potentially really beneficial. Maybe if you learned more about it you too could enjoy its benefits.
Although most developers and users are still feeling their way through Hadoop and (more specifically MapReduce), the truth is Google wrote that paper in 2004. That’s ten years ago! Million Dollar Baby won Best Picture that year. Yeah! by Usher, Lil John and Ludacris topped the charts in the United States. And Facebook had only just started to kill work productivity and violate your privacy. As long ago as that feels, it is an eternity in technology.
Let me first say I love Java. There is a reason it’s the most popular programming language in the world. For me personally, I made a career out of building systems in Java, and I even teach a course in Java. But when it comes to Big Data, Java simply doesn’t cut it. Everybody knows functional languages have enjoyed a renaissance as Big Data has become a thing. And for good reason: immutable (or mostly immutable) state, lazy evaluation, the natural fit with recursion, and so on.