Data Engineer

Senior Data Engineer Senior Data Engineer to fully support the ongoing predictive modeling work by optimizing the data, creating and maintaining data pipelines and ETL processes and working with a Master Data Management (MDM) system implementation for enterprise-wide data distribution. Responsibilities Take ownership of all data assets with accountability for tuning, optimization and scalability (pipelines and ETL processes) Plan, design, operate and troubleshoot databases in collaboration with the DBA Experimental design ndash proof of concept (POC), systems reliability, performance and design of streamlined data delivery Conduct data analysis and ensure transformation covers all possibilities including data irregularities Implement and maintain a vetted master data management (MDM) system Keep abreast of the most cutting-edge developments in this rapidly evolving field Work in conjunction with a close-collaborating, newly formed team working on highly innovative and cutting-edge concepts and processes Qualifications Experience and firm knowledge of managing multiple disparate data sources, ETL and data pipelines for machine learning and BI Experience documenting data requirementsdata stories and maintenance of data models Excellent knowledge of SQLT-SQL and Python Experience with cloud platforms, particularly Amazon Web Services (AWS) Advanced knowledge of relational database systems as well as Columnar Databases and NoSQL Knowledge and experience with data modeling, data warehousing, information systems Software Engineering (Agile, DevOps, Design Techniques) Distributed systems and parallel programming Knowledge of master data management (MDM) and experience maintaining data catalogs Preferred knowledge of SQL Server, Pyspark, Glue and Data Lake design and architecture Exposure to Big Data Architecture in AWS, EMR, S3, Redshift, Athena, RDS, DynamoDB and Quicksight Basic knowledge of Machine LearningDeep learning is a big plus Data security best practices

source https://www.jobsinmiramar.com/it-tech-support/data-engineer-ad426bc/

source https://jobsinmiramar.tumblr.com/post/614809606476251136