Senior AI Platform & Data Engineer

About TCS:

A purpose-led organization that is building a meaningful future through innovation, technology, and collective knowledge. We're #BuildingOnBelief.

Tata Consultancy Services (TCS) is a global leader in IT services, digital and business solutions that partners with its clients to simplify, strengthen and transform their businesses. TCS offers a consulting-led, integrated portfolio of IT, BPS, infrastructure, engineering and assurance services. We ensure the highest levels of certainty and satisfaction through a deep-set commitment to our clients, comprehensive industry expertise and a global network of innovation and delivery centers. For more information, visit us at www.tcs.com.

Role Overview

We are seeking a highly skilled and versatile Senior AI Platform & Data Engineer to join our team via Managed Services. In this role, you will act as a critical builder of our Universal Data Platform (UDP), bridging the gap between robust Data Engineering, Analytics Engineering, and Generative AI enablement.

Your primary objective is to integrate diverse enterprise data sources and build reliable, automated data pipelines that feed a structured, self-service data catalog and knowledge base. You will ensure that company-wide data is not only clean and well-governed but also strictly monitored for anomalies (such as missing null values or schema drift) so that it remains optimized for agentification—enabling AI Agents, Large Language Models (LLMs), and Retrieval-Augmented Generation (RAG) workflows to securely and seamlessly consume trusted enterprise knowledge.

Required Qualifications & Skills:

• Bachelor's Degree in Computer Science, Data Science, Information Technology, Computer Engineering, Software Engineering, Artificial Intelligence, or a related field.
• Minimum 6-10 years of experience in Data Engineering, AI Engineering, Machine Learning Engineering, Analytics Engineering, or related fields.
• Experience: 4+ years of experience as a Data Engineer, AI Engineer, or Analytics Engineer, ideally within environments emphasizing big data integration, platform enablement, or AI product development.
• Programming Languages: Exceptional proficiency in Python and SQL.
• Data Quality & Monitoring Tools:
• Practical experience with data profiling, testing, and anomaly detection frameworks (e.g., Great Expectations, Soda, dbt tests).
• Familiarity with data observability and pipeline monitoring suites to track data completeness (null checks, range boundaries, type assertions).

Key Responsibilities

• Universal Data Platform Integration: Collaborate on the design and execution of UDP ingestion pipelines, integrating various legacy and modern source systems to establish a unified data ecosystem.
• Generative AI & LLM Implementation: Lead experimentation and development of GenAI solutions, including Knowledge Bases, Chatbots (e.g., Copilots), text-to-SQL workflows, and document summarization using state-of-the-art frameworks.
• Pipeline Observability & Monitoring: Implement and manage specialized data pipeline monitoring platforms (e.g., Monte Carlo, Datadog, Prometheus, Soda) to track pipeline health, execution times, failures, and maintain high platform uptime.
• Data Quality & Anomaly Detection: Set up automated data anomaly detection systems to identify data discrepancies, run-time schema drift, missing metadata, and critical data gaps (such as unexpected null values or zero-records).
• Knowledge Base & Data Cataloging: Build and maintain proper data catalogs, metadata schemas, and centralized knowledge bases, enabling anyone in the company to easily discover, manage, and own their business domain's data.
• Enablement for Agentification: Structure and optimize corporate data assets specifically for AI agent consumption. This includes establishing proper indexing, document chunking pipelines, and preparing unstructured/structured data for high-performance semantic retrieval.
• Generative AI & Semantic Search Engineering: Design and implement robust RAG pipelines, semantic search layers, and vector databases (e.g., Elasticsearch, Weaviate, Qdrant) to connect the enterprise data platform with downstream LLMs.

Generative AI & Search Tech Stack:

• Solid understanding of vector search concepts and databases (Weaviate, Qdrant, Elasticsearch, or Vertex AI Vector Search).
• Experience with LLM orchestration, RAG pipelines, and agent frameworks (LangChain, LangGraph, Dify, LlamaIndex).

Cloud & Platform Infrastructure:

• Experience with Google Cloud Platform (GCP) services (BigQuery, Vertex AI, GKE, CloudRun).
• Strong understanding of containerization (Docker, Kubernetes) and deploying data/AI applications as scalable APIs.
• Data Visualization & Prototyping: Ability to quickly spin up interactive dashboards or internal portals (Streamlit, Metabase, PowerBI) to demonstrate data cataloging, quality metrics, and agent capabilities to stakeholders.

Ideal Candidate Profile

• Data Quality Champion: You treat data quality as a product priority, knowing that AI agents and downstream applications are only as good as the underlying data inputs (preventing Garbage In, Garbage Out).
• Platform Thinker: You design reusable, modular pipelines, templates, and metadata schemas that empower the entire organization to easily discover and self-serve data.
• AI-Native Mindset: You understand how LLMs and agents think and actively structure enterprise knowledge bases to be easily parsed and queried by AI systems.
• Collaboration Champion: You can work closely with business units to help them organize, catalog, monitor, and truly own their domain's data.