Neudesic -an IBM company – Data Engineer
Job Description
• Analyze business problems and design technology solutions
• Ability to understand the data transformation requirements and implement the same in an optimum solution using Azure data services
• Capable to participate in customer discussion and suggest solutions as necessary
• Ability to carry out the data modeling activities and deliver the solution in agreement with baselined time & quality
• Stay focused on-task completion and delivery
• Supporting existing processes and implementing change requirements as part of a structured change control process
• Working in a project environment, communicating any identified risks and issues to the project manager accordingly and providing inputs to the change control process
• The consultant need to be hands-on and highly technical. Consultant will be in a customer facing role and accountable for the end-to-end technology delivery and deployment experience of data pipelines using Azure data services.
• Perform data cleansing, data validation, sanity checking, and unit testing associate with it
• Perform production cutover activities from a technical perspective
Eligibility
Must Have Skills:
• Prior experience in ETL, data pipelines, data flow techniques using Azure Data Services
• Working experience in Python, Scala, Py Spark, Azure Data Factory, Azure Data Lake Gen2, Databricks, Azure Synapse and file formats like JSON & Parquet
• Experience in creating ADF Pipelines to source and process data sets.
• Experience in creating Databricks notebooks to cleanse, transform and enrich data sets.
• Good understanding about SQL, Databases, NO-SQL DBs, Data Warehouse, Hadoop and various data storage options on the cloud.
• Development experience in orchestration of pipelines
• Experience in deployment and monitoring techniques
• Working experience with Azure DevOps CI/CD pipelines to deploy Azure resources.
• Experience in handling operations/Integration with source repository
• Must have good knowledge on Datawarehouse concepts and Datawarehouse modelling
Good to Have Skills:
• Familiarity with DevOps, Agile Scrum methodologies and CI/CD
• Domain-driven development exposure
• Analytical / problem solving skills
• Strong communication skills
• Good experience with unit, integration and UAT support
• Able to design and code reusable components and functions
• Should able to review design, code & provide review comments with justification
• Zeal to learn new tool/technologies and adoption
• Power BI and Data Catalog experience
About Company
A global professional services firm helping organizations navigate the intersection between people, technology, and business.
https://www.neudesic.com/
