As a Data Engineer - Big Data and ETL, you will participate in
all aspects of the software development lifecycle which includes estimating,
technical design, implementation, documentation, testing, deployment and
support of applications developed for our business partners. As a member
working in a team environment you will take direction from solution/Platform
architects and Leads on development activities. Work inside a team with Strong
Data Warehousing skills, including: Data cleanup, ETL, ELT and handling
scalability issues for enterprise level data warehouse. Implements,
troubleshoots, and optimizes distributed solutions based on modern big data
technologies like Hive, Hadoop, Spark, Elastic Search, Storm, Kafka,
etc. in both an on premise and cloud deployment model to solve large scale
processing problems. The software application landscape within KP-IT spans many
technologies that include:
building data platform on cloud (AWS or Azure)
using Python, Java or any other language to solving data
implementing SDLC best practices and Agile methods
In addition to the responsibilities listed below, this position is responsible for planning, designing, and building database systems that are stable, recoverable, and integrated with other available technology stacks for the purpose of efficient, secure data utilization. This includes understanding the interoperability between databases and other dependent technology stacks (performance scalability, stability, capacity planning, etc.), possessing in-depth knowledge in one dependent technology stack (e.g. windows admin, networking, storage, etc.), negotiating with other engineering platform partners, and consulting as 2nd level technical expert on specific platforms. This role also requires familiarity with at least one other DBMS platform in addition to specialty and administrative skills in at least one other platform.Some of the unique challenges this position will face include database considerations for a large, corporate enterprise and a high degree of complexity and non-uniformity.
Conducts or oversees business-specific projects by applying deep expertise in subject area; promoting adherence to all procedures and policies; developing work plans to meet business priorities and deadlines; determining and carrying out processes and methodologies; coordinating and delegating resources to accomplish organizational goals; partnering internally and externally to make effective business decisions; solving complex problems; escalating issues or risks, as appropriate; monitoring progress and results; recognizing and capitalizing on improvement opportunities; evaluating recommendations made; and influencing the completion of project tasks by others.
Practices self-leadership and promotes learning in others by building relationships with cross-functional stakeholders; communicating information and providing advice to drive projects forward; influencing team members within assigned unit; listening and responding to, seeking, and addressing performance feedback; adapting to competing demands and new responsibilities; providing feedback to others, including upward feedback to leadership and mentoring junior team members; creating and executing plans to capitalize on strengths and improve opportunity areas; and adapting to and learning from change, difficulties, and feedback.
As part of the IT Engineering job family, this position is responsible for leveraging DEVOPS, and both Waterfall and Agile practices, to design, develop, and deliver resilient, secure, multi-channel, high-volume, high-transaction, on/off-premise, cloud-based solutions.
Provides consultation and expert technical advice on IT infrastructure planning, engineering, and architecture for assigned systems by assessing the implications of IT strategies on infrastructure capabilities.
Provides some recommendations and input on options, risks, costs, and benefits for systems designs.
Leverages partnerships with IT teams and key business partners to troubleshoot complex systems.
Serves as a functional expert and collaborates with architects and software engineers to ensure functional specifications are converted into flexible, scalable, and maintainable system designs.
Translates business requirements, and functional and non-functional requirements, into technical specifications that support integrated and sustainable designs for complex or high impact infrastructure systems by partnering with Business Analysts to understand business needs and functional specifications.
Ensures system designs adhere to company architecture standards.
Builds partnerships with counterparts in various IT Teams (e.g., database, operations, technical support) throughout system development and implementation.
Serves as a technical expert for project teams throughout the implementation and maintenance of assigned enterprise infrastructure systems by defining and overseeing the documentation of detailed standards (e.g., guidelines, processes, procedures) for the introduction and maintenance of services.
Mentors other technical resources throughout infrastructure systems development.
Reviews and validates technical specifications and documentation for complex or multi-dimensional solutions.
Leads the development and modification of solutions by identifying technical solutions to business problems.
Collaborates with business leaders, Solutions, and lead enterprise architects to review business drivers, and establish a foundation for enterprise systems planning.
Reviews benchmarking results, and provides information to support current and future infrastructure needs and projects to IT leadership. Provides preliminary conclusions.
Benchmarks and evaluates IT trends and technologies to identify opportunities and considerations that impact ROI.
Makes recommendations on resources required to maintain service levels and meet new demands.
Guides and drives physical architecture design for new initiatives. khgrsr
Minimum four (4) years experience in the planning, design, and implementation of security solutions, including Minimum two (2) years experience leading the design, implementation, troubleshooting, and operation of security technologies.
Minimum two (2) years in a technical leadership role with or without direct reports.
Bachelor's degree in Computer Science, CIS, or related field and Minimum eight (8) years experience in an IT operations environment with technical experience in distributed technologies, systems development, and/or networking. Additional equivalent work experience may be substituted for the degree requirement.
Three (3) years of experience building technology solutions to meet corporate or industry IT regulatory requirements.
Three (3) years experience in the design and implementation of complex data infrastructure solutions.
Three (3) years experience in IT infrastructure consulting.
California,Pleasanton,Pleasanton Tech Cntr Building F 5810 Owens Dr.
Scheduled Weekly Hours:
Mon, Tue, Wed, Thu, Fri
Working Hours Start:
Working Hours End:
Employee Group/Union Affiliation:
Salaried, Non-Union, Exempt
IT ENG Infrastructure
Yes, 5 % of the Time