Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 14 hours
Course Outline
Introduction to Multimodal LLMs in Vertex AI
- Exploring the scope of multimodal capabilities within Vertex AI
- Overview of Gemini models and their supported modalities
- Key applications in enterprise and research environments
Configuring the Development Environment
- Setting up Vertex AI for multimodal workflow execution
- Managing datasets spanning various modalities
- Lab session: Environment initialization and dataset preparation
Long Context Windows and Advanced Reasoning
- Concepts behind long-context workflow processing
- Applications in strategic planning and decision-making
- Lab session: Implementing long-context data analysis
Designing Cross-Modal Workflows
- Synthesizing analysis from text, audio, and image sources
- Orchestrating multimodal steps within integrated pipelines
- Lab session: Architecting a comprehensive multimodal pipeline
Managing Gemini API Parameters
- Configuration of multimodal input and output streams
- Strategies for optimizing inference speed and operational efficiency
- Lab session: Fine-tuning Gemini API parameters for optimal results
Advanced Applications and System Integrations
- Development of interactive multimodal agents and assistants
- Connecting external APIs and third-party tools
- Lab session: Constructing a functional multimodal application
Evaluation and Continuous Improvement
- Techniques for testing multimodal system performance
- Establishing metrics for accuracy, alignment, and drift detection
- Lab session: Conducting comprehensive workflow evaluations
Summary and Future Directions
Requirements
- Strong proficiency in Python programming
- Hands-on experience in machine learning model development
- Understanding of multimodal data types (text, audio, images)
Intended Audience
- AI Researchers
- Advanced Developers
- Machine Learning Scientists
