AWS Data Engineer

Job Title:                            Data Engineer

Reporting To:                   Data Engineering Lead  

Business:                          Technology and Innovation – Data Engineering

Location:                              India

Lloyd’s List Intelligence has proven success across the globe with offices in APAC (Singapore, China, Hong Kong and Japan), EMEA (UK, UAE) and USA. We now look to further expand our global footprint and add a location that gives us the capability to build teams of highly talented colleagues and deliver growth in our Technology, Data and Product functions.

Role Profile

We are seeking a Data Engineer to join our team of existing Data Engineers to assist with the ongoing maintenance and innovation of our data processing and ingestion pipelines and systems.

The successful candidate will join a team of engineers responsible for delivering project deliverables involving back-end development, maintenance of our data ingestion, streaming, and analytical systems.  The team is tasked with ensuring our various data ingestion systems achieve a high level of reliability and work closely with our product, data science and software engineering teams to support existing and new data workflows.

This is a full-time permanent role reporting to the Data Engineering Lead. This role is balanced working in our Chennai office.

Key Responsibilities

• Collaborate with Product teams ensuring that raw data is cleansed and transformed and useable by downstream consumers (ML Engineers, BI analytics)
• Assist and advise on the re-development and modernisation of end-to-end ETL pipelines and introduce new technologies where appropriate in a real-time streaming environment dealing with very large data volumes
• Working with AWS cloud infrastructure (specifically SQS, SNS, Redshift, OpenSearch, Athena, Kinesis, AWS code pipeline) to develop, innovate and maintain data flowing through various queues and data warehouses
• Working with a variety of data repository platforms (including SQL stores such as Oracle), as well as implementing data visualisation and network analysis (e.g GraphDB)
• Maintain and ‘productionise’ machine learning and AI models
• Assist in the creation of next generation data ingestion platforms – sourcing data using Webscrapes, APIs, Email, and flat file (FTP) methods
• Understanding conflict resolution methods and assist subject matter experts in the debugging on data ingestion and managing overall feed uptimes across a large set of data collectors.
• Create and maintain detailed documentation and functional design specifications including data flows and data conversion
• Provide technical information to assist in the development of client facing product documentation
• Adhere to change management protocols and version control
• Present advanced technical designs with non-technical stakeholders

What we are looking for:

Essential

• Bachelor’s honours degree preferably in computer science, or engineering field, or demonstrable technical experience in a related engineering discipline 

• At least 3 years of ETL experience using flat files to transposed Data Models 

• Experience in troubleshooting and debugging Python, SQL (Oracle & other SQL platforms), SNS, SQS, Kinesis and AWS Lambda 

• 2 years’ experience with Data Mining and Analytical skills required to encode sophisticated business transformation logic in a pipeline and serve predictive models 

• Familiarity with the design, development, and maintenance of best-in-class BI capabilities, including data warehouse data structures and data pipelines (Redshift, Athena etc) 

• Experience working with Agile development process 

• Proven analytical thinking ability, ease with quantitative analysis, and excellent problem-solving skills 

• Self–discipline and willingness to learn 

• Ability to make connections with technical and non-technical stakeholders 

• Excellent verbal and written communication skills 

Desirable

• Domain knowledge or experience working within the maritime or an adjacent industry
• Experience of streaming or ‘real-time’ processing at high data volumes
• Previous experience using JAVA and C#
• Familiarity of architectural design patterns for micro-services leveraging relational and big data technologies
• Demonstrable experience working with Big Data environments
• Experience of the SAS Analytics platform of particular interest

We know that sometimes the 'perfect candidate' doesn't exist, and people can be put off from applying for a job if they don't fit all the requirements. If you're excited about working for us and have most, but not all, of the skills or experience we're looking for, please go ahead and apply. You could be just what we need!

We believe strongly in the value of diversity and in creating supportive, inclusive environments where our colleagues can succeed. As such, we are proud to be an Equal Opportunity Employer. We do not discriminate based on race, color, ancestry, national origin, religion, mental or physical disability, medical condition, genetic information, sex (including pregnancy, childbirth, and related medical conditions), sexual orientation, gender identity, gender expression, age, marital status, military or veteran status, citizenship, or other protected characteristics under federal, state, or local law.

Please let us know of any adjustments we need to make for you to show us your best self.

About us

We are the industry experts delivering actionable maritime insight, data, technology, and analytics trusted by 60,000 professionals across the globe to drive commercial advantage, evaluate risk, and support the efficient and lawful movement of seaborne trade.
 
Our data is collected only from the most reputable partners and accurate sources, many of which are exclusive to us and our customers.
 
Our advanced analytics, artificial intelligence, and industry expertise transform our unparalleled data into powerful insight delivered through data and analytics platforms and services, news and commentary, and publications that help professionals in maritime operations, risk, and compliance stay well informed.
 
Leading seaborne trade through change with data transparency, innovative technology, and human ingenuity.

Our Vales and Behaviours

For LLI, these aren’t just the wallpaper. These are the behavioural expectations led and role modelled by our CEO Michael Dell, and they are also the design principles of an change or initiative we put in place.

Build trust & Transparency

Transparency and trust needs to be at the heart, with commitment to honest communication about

individual progress, as well as company news, regular townhall meetings, clear messaging about the

company's direction, and transparency in addressing internal challenges such as gender and race

inequality and recognition/reward.

Embrace collective brilliance

Collaboration and teamwork are the foundation of our collective success. Our data highlights the importance of

working together cross-company and communicating with agility to solve problems. There is a strong emphasis

on friendly, authentic relationships and courageous collaboration.

Work with respect

Respect for one another's well-being, fairness in recognition of effort, inclusivity in approach, and

celebration of cultural differences is critical. There is emphasis placed on social awareness and

positive impact.

Take responsibility

The importance of being accountable to each other cannot be underestimated, recognizing each

other's work, and maintaining a challenge culture to ensure credibility, dependability, and

trustworthiness. There is focus on stepping up to take on more if it helps the collective organization.

Simplify the complex

There is a focus in our data on delivering excellence, recognizing and learning from failures, and

a commitment to delivering work that puts customers first and provides market-leading

expertise.

Move fast & be bold

The data sources emphasize the need to embrace change in an agile way, with a positive approach to

challenges and transformation. It includes a desire to evolve and remain industry leaders, embracing

challenges and promoting a culture of curiosity and continuous learning.