In-Demand Job Profiles in Big Data and Analytics Industry

The Big Data and Analytics field is constantly evolving and constantly making way for new technologies to be picked up and mastered for companies and professionals to stay ahead in the competition. As this field is further broken down in terms of skill requirements, there are many interdependent roles that emerge and the definitions of these roles are also dynamically evolving. In this data-driven era, not just the top players in every industry but also most of the small and mid-size businesses (SMBs) are spending their bucks on hiring different Big Data and Analytics related talents. Job portals globally have seen a significant hike in Big Data related postings in the past few years. So, whether you are a fresher or someone who is now looking for a change in your career path, the basic understanding of different profiles can better guide you as to in which domain lies your interest and expertise. ‘Big Data’ is a rather abstract term with several diverse positions under its umbrella. The profiles can be broadly classified into categories:

• Big Data Engineers
• Big Data Analysts

Although stated distinctly, there are many skills that are common to both. In simple words, if you are a mathematician or statistician with a liking for programming and analysis then the latter can be your prospects and if you are proficient in programming skills, databases, etc. then it could be the former. To go by an interesting definition given by Josh Wills, Director of Data Engineering at Slack, a data scientist/engineer (in general) is a person who is better at statistics than any software engineer and better at software engineering than any statistician. Here are six careers paths under Big Data and Analytics. Let’s take a closer look at what each position stands for and the tools used in each: Database Manager: It is a Database management role wherein they oversee the development and use of data systems. Essentially an IT role, it may be required for database managers to perform some data analysis. They are tech-savvy individuals who also possess impressive communication skills, although they might not be required to have deep programming knowledge. They monitor the data systems and the underlying infrastructure and are upbeat with new technologies. Although a separate job role, a database administrator is akin to the database manager. A database admin looks over the management of the company’s databases. Key skills required:

• NoSQL Databases
• SQL ecosystem
• Apache Hadoop, MapReduce, HDFS
• Querying tools such as Spark, Pig, Hive

Salary range (2017): $122,250–$177,000 Projected job growth (2016–2017): 3.7% Data Engineer– A role that is often mistaken with that of a Data Scientist. According to a report by Bureau of Labor Statistics, the demand for Data engineers is subject to grow 15% by 2024 in the US alone. They are non-analysts and are supposed to be middlemen between business analysts/executives and data scientists. At the very base, the technologies used by both data engineers and managers are the same but differ in their levels of implementation. They make sure the team is working towards the desired business goals and is well-versed with the know-hows of the technologies. Key skills required:

• Hadoop-based technologies like MapReduce, Hive
• SQL based technologies like PostgreSQL and MySQL
• Teradata, Amazon Redshift, Microsoft Data Warehouse
• NoSQL technologies like Cassandra and MongoDB

Salary range (2017): $135,000–$196,000 Projected job growth (2016–2017): 5.8% Data Architect – They work closely with data engineers in developing and designing databases and know the complete architecture. Upon designing the databases, they work with the programmers, engineers, analysts for implementation of these databases. Key skills required:

• Oracle, Microsoft SQL server, MySQL and RDBMS
• ETL frameworks
• Business Objects software, XML, SOA, ETL, BI, ERP, and Agile

Salary range (2017): $131,250–$184,000 Projected job growth (2016–2017): 4.1% Business Analyst: Business Analyst is a term with an amalgam of diverse duties and skills required. But, in simple terms, it is an analyst role wherein they collect, analyse historical and current data and develop cost-effective solutions. It includes Business Intelligence and Data warehousing under its hat and hence possesses data analysis expertise. Their roles include setting up reporting tools, querying of databases and needs knowledge of Big Data frameworks. Business Analyst also provides modelling of interactive data visualisations. Many enterprises also hire a BI Specialists that work in the BI framework Support. There is a thin demarcation in terms of the roles of BA and BI but they have the same underlying working; they both focus on optimising their business outcomes. Key skills required:

• NoSQL databases
• RDBMS
• Data warehousing tools
• Tableau, QlikView
• SAP, etc.

Salary range (2017): $118,000–$171,750 Projected job growth (2016–2017): 4.3% Machine Learning Engineer– An ML expert is responsible for developing or utilizing algorithms to produce automated systems. These algorithms help in exploring patterns using predictive statistical analysis and hence, this role requires a good knowledge of statistics and programming. The final output for an ML engineer is a working software that runs without much of human intervention. For example, a program that understands the overall customer reaction of a product and gives appropriate predictions that help in business development. Key skills required:

• Programming languages (R, Python, etc.)
• Maths and Statistics
• Computer Science fundamentals

Salary range (2017): $142,500–$190,050 Projected job growth (2016–2017): 4.8% Data Scientist– This position is seen as Data- oriented professional that is concerned with pulling insights from the data using appropriate tools. A data scientist is considered the ‘Jack of all trades’ as he possesses a blend of business and technical skills from analysis to data wrangling. He also needs to possess sound communication skills to translate the findings to the non-technical business executives which can be a daunting task. Key skills required:

• Programming
• Maths and Statistics
• Communication skills
• Databases

Salary range (2017): $111,600–$163,500 Projected job growth (2016–2017): 6.4% This article covers the main positions pertaining to Big data and Analytics and serves as an introductory guide for professionals trying to break into this field. At many companies, employees hold a combination of the above-mentioned job profiles thus taking up multiple roles which may be addressed with different names. It is important to note that this field will be constantly evolving with technology advancements. Hence, it is necessary to be updated with the current trends and technologies.