Tibor Kiss

You can never be sure when you’ll be the target of a distributed denial-of-service (DDoS) attack. For investigative journalist Brian Krebs, that day came on May 12, when his site KrebsOnSecurity experienced one of the largest DDoS attacks seen to date.

At 6.3 terabits per second (Tbps), or roughly 63,000 times the speed of broadband internet in the U.S., the attack was 10 times the size of the DDoS attack Krebs faced in 2016 from the Mirai botnet. That 2016 incident took down KrebsOnSecurity.com for four days, and was so severe that his then-DDoS protection service asked him to find another provider, Krebs said in his report on the May attack.

Following the 2016 incident, Krebs signed up for Project Shield, a free Google service that offers at-risk, eligible organizations protection against DDoS attacks. Since then, his site has stayed reliably online in the face of attacks — including the latest incident.

Organizations in eligible categories, including news publishers, government elections, and human rights defenders, can use the power of Google Cloud’s networking services in conjunction with Jigsaw to help keep their websites available and online. 

Project Shield acts as a reverse proxy service — customers change their DNS settings to send traffic to an IP address provided by Project Shield, and configure Project Shield with information about their hosting server. The customer retains control over both their DNS settings and their hosting server, making it easy to enable or disable Project Shield at any time with a simple DNS switch.

Built on the strength of Google Cloud networking services, including Cloud Load Balancing, Cloud CDN, and Cloud Armor, Project Shield’s services can be configured through the Project Shield dashboard as a managed experience. This solution works together to mitigate attacks and serve cached content from multiple points on Google’s edge network. It’s a combination that has protected KrebsOnSecurity before, and has successfully defended many websites against some of the world’s largest DDoS attacks.

In the May incident against Krebs, the attack was filtered instantly by Google Cloud’s network. Requests for websites protected by Project Shield pass through Google Cloud Load Balancing, which automatically blocks layer 3 and layer 4 volumetric DDoS attacks. 

In the May incident, the attacker sent large data packets to random ports at a rate of approximately 585 million packets per second, which is over 1,000 times the usual rate for KrebsOnSecurity.

Cloud Armor, which embeds protection into every load balancer deployment, blocked the attack at the load balancing level because Project Shield sits behind the Google Cloud Load Balancer, which proxies only HTTP/HTTPS traffic. Had the attack occurred with well-formed requests (such as at Layer 7, also known as the application layer), additional defenses from the Google Cloud global front end would have been ready to defend the site. 

Cloud CDN, for example, makes it possible to serve content for sites like KrebsOnSecurity from cache, lessening the load on a site’s servers. Cloud Armor would have actively filtered incoming requests for any remaining traffic that may have bypassed the cache to allow only legitimate traffic through. 

Additionally, Cloud Armor’s Adaptive Protection uses real-time machine learning, which helps identify attack signatures and dynamically tailor rate limits. These rate limits are actively and continuously refined, allowing Project Shield to harness Google Cloud’s capabilities to mitigate almost all DDoS attacks in seconds.

Project Shield defenses are automated, with no customer defense configuration needed. They’re optimized to capitalize on the powerful blend of defensive tools in Google Cloud’s networking arsenal, which are available to any Google Cloud customer. 

As KrebsOnSecurity and others have experienced, DDoS attacks have been getting larger, more sophisticated, and more frequent in recent years. Let the power and scale of Google Cloud help protect your site against attacks when you least expect them. Eligible organizations can apply for Project Shield today, and all organizations can set up their own Cloud Networking configuration like Project Shield by following this guide.

Developers love Cloud Run, Google Cloud’s serverless runtime, for its simplicity, flexibility, and scalability. And today, we’re thrilled to announce that NVIDIA GPU support for Cloud Run is now generally available, offering a powerful runtime for a variety of use cases that’s also remarkably cost-efficient. 

Now, you can enjoy the following benefits across both GPUs and CPUs:

• Pay-per-second billing: You are only charged for the GPU resources you consume, down to the second.

• Scale to zero: Cloud Run automatically scales your GPU instances down to zero when no requests are received, eliminating idle costs. This is a game-changer for sporadic or unpredictable workloads.

• Rapid startup and scaling Go from zero to an instance with a GPU and drivers installed in under 5 seconds, allowing your applications to respond to demand very quickly. For example, when scaling from zero (cold start), we achieved an impressive Time-to-First-Token of approximately 19 seconds for a gemma3:4b model (this includes startup time, model loading time, and running the inference)

• Full streaming support: Build truly interactive applications with out-of-the box support for HTTP and WebSocket streaming, allowing you to provide LLM responses to your users as they are generated.

Support for GPUs in Cloud Run is a significant milestone, underscoring our leadership in making GPU-accelerated applications simpler, faster, and more cost-effective than ever before.

“Serverless GPU acceleration represents a major advancement in making cutting-edge AI computing more accessible. With seamless access to NVIDIA L4 GPUs, developers can now bring AI applications to production faster and more cost-effectively than ever before.” – Dave Salvator, director of accelerated computing products, NVIDIA

AI inference for everyone

One of the most exciting aspects of this GA release is that Cloud Run GPUs are now available to everyone for NVIDIA L4 GPUs, with no quota request required.This removes a significant barrier to entry, allowing you to immediately tap into GPU acceleration for your Cloud Run services. Simply use --gpu 1 from the Cloud Run command line, or check the “GPU” checkbox in the console, no need to request quota:

Production-ready

With general availability, Cloud Run with GPU support is now covered by Cloud Run’s Service Level Agreement (SLA), providing you with assurances for reliability and uptime. By default, Cloud Run offers zonal redundancy, helping to ensure enough capacity for your service to be resilient to a zonal outage; this also applies to Cloud Run with GPUs. Alternatively, you can turn off zonal redundancy and benefit from a lower price for best-effort failover of your GPU workloads in case of a zonal outage.

Multi-regional GPUs

To support global applications, Cloud Run GPUs are available in five Google Cloud regions: us-central1 (Iowa, USA), europe-west1 (Belgium), europe-west4 (Netherlands), asia-southeast1 (Singapore), and asia-south1 (Mumbai, India), with more to come.

Cloud Run also simplifies deploying your services across multiple regions. For instance, you can deploy a service across the US, Europe and Asia with a single command, providing global users with lower latency and higher availability. For instance, here’s how to deploy Ollama, one of the easiest way to run open models, on Cloud Run across three regions:

See it in action: 0 to 100 NVIDIA GPUs in four minutes

You can witness the incredible scalability of Cloud Run with GPUs for yourself with this live demo from Google Cloud Next 25, showcasing how we scaled from 0 to 100 GPUs in just four minutes.

Unlock new use cases with NVIDIA GPUs on Cloud Run jobs

The power of Cloud Run with GPUs isn’t just for real-time inference using request-driven Cloud Run services. We’re also excited to announce the availability of GPUs on Cloud Run jobs, unlocking new use cases, particularly for batch processing and asynchronous tasks:

• Model fine-tuning: Easily fine-tune a pre-trained model on specific datasets without having to manage the underlying infrastructure. Spin up a GPU-powered job, process your data, and scale down to zero when it’s complete.

• Batch AI inferencing: Run large-scale batch inference tasks efficiently. Whether you’re analyzing images, processing natural language, or generating recommendations, Cloud Run jobs with GPUs can handle the load.

• Batch media processing: Transcode videos, generate thumbnails, or perform complex image manipulations at scale.

What Cloud Run customers are saying

Don’t just take our word for it. Here’s what some early adopters of Cloud Run GPUs are saying:

“Cloud Run helps vivo quickly iterate AI applications and greatly reduces our operation and maintenance costs. The automatically scalable GPU service also greatly improves the efficiency of our AI going overseas.” – Guangchao Li, AI Architect, vivo

“L4 GPUs offer really strong performance at a reasonable cost profile. Combined with the fast auto scaling, we were really able to optimize our costs and saw an 85% reduction in cost. We’ve been very excited about the availability of GPUs on Cloud Run.” – John Gill at Next’25, Sr. Software Engineer, Wayfair

“At Midjourney, we have found Cloud Run GPUs to be incredibly valuable for our image processing tasks. Cloud Run has a simple developer experience that lets us focus more on innovation and less on infrastructure management. Cloud Run GPU’s scalability also lets us easily analyze and process millions of images.” – Sam Schickler, Data Team Lead, Midjourney

Get started today

Cloud Run with GPU is ready to power your next generation of applications. Dive into the documentation, explore our quickstarts, and review our best practices for optimizing model loading. We can’t wait to see what you build!

In today’s rapidly evolving technological landscape, artificial intelligence (AI) stands as a transformative force, reshaping industries and redefining possibilities. Recognizing AI’s potential and leveraging its data landscape on Google Cloud, Magyar Telekom, Deutsche Telekom’s Hungarian operator,  embarked on a journey to empower its workforce with AI knowledge and tools. This endeavor led to the creation of Pluto AI — an internal AI platform that has grown into a comprehensive framework for diverse AI solutions. 

As one of Hungary’s largest telecommunications operators, Magyar Telekom’s ultimate vision is to embed AI into every aspect of its operations, empowering every employee to leverage AI’s potential. Pluto AI is a significant step toward achieving this goal, fostering a culture of innovation and data-driven decision-making.

Magyar Telekom’s leadership recognized that AI proficiency is now essential for future success. However, the company faced challenges, including employees with varying levels of AI understanding and a lack of accessible tools for experimentation and practical application. As a result, Magyar Telekom aimed to democratize AI knowledge and foster a culture of experimentation by building a scalable solution that could adapt to its evolving AI needs and support a wide range of use cases. 

To enable business teams across Magyar Telekom to utilize generative AI, the Pluto AI team developed a simple tool that provided a safe and compliant way to prompt large language models (LLMs). They also created educational content and training for business teams on how to use gen AI and what opportunities it brings. This approach provided other teams with the building blocks to quickly construct the AI solutions they needed.

With Pluto AI, Magyar Telekom spearheaded the successful adoption of gen AI across the company, quickly expanding the platform to support additional use cases without the need for the central platform team to have a deep understanding of them.

Developing Pluto AI

Magyar Telekom’s AI Team partnered with Google Cloud Consulting to accelerate the development of Pluto AI. This collaboration ensured that the platform was built on best practices, aligned with industry standards, and met security and compliance requirements of a regulated industry.

Here are some of the key features and functionality of Pluto AI: 

1. Modular framework

Pluto AI’s modular architecture allows teams to seamlessly integrate, change, and update various AI models, tools and various architectural patterns. This flexibility enables the platform to cater to a wide range of use cases and rapidly evolve alongside Magyar Telekom’s AI strategy.

The core modules of Pluto AI include:

• Large language models: Pluto AI integrates with state-of-the-art LLMs, enabling natural language understanding, text and image generation, and conversational AI applications.

• Code generation and assistance: The platform supports code generation, autocompletion, and debugging, boosting developer productivity and code quality. Pluto AI provides both a coding model, accessible via its user interface, for all development levels and IDE integration for experienced coders.

• API: Pluto AI’s models can be called via API, enabling all parts of Magyar Telekom to utilize and integrate AI capabilities into their existing and new solutions.

• Retrieval augmented generation (RAG) with grounding capabilities: RAG combines LLMs with internal knowledge sources, including multimodal content like images and videos. This enables teams to build AI assistants that can access and synthesize information from vast datasets and add evidence like extended citations from both corporate and public data to their responses.

• Customizable AI assistants: Users can create tailored, personalized AI assistants by defining system prompts, uploading documents, and fine-tuning model behavior to meet their business needs.

2. Technical implementation

Pluto AI runs on Compute Engine using virtual machines, providing scalability, reliability, and efficient resource management. The platform also utilizes foundation models from the Model Garden on Vertex AI, including Google’s Gemini, Imagen, and Veo models, Anthropic’s Claude 3.5 Sonnet, and more. Magyar Telekom also deployed ElasticSearch on Google Cloud to store the knowledge bases necessary for enabling RAG workflows. 

In addition to these core components, Pluto AI also utilizes other Google Cloud services to help develop production-ready applications, such as Cloud Logging, Pub/Sub, Cloud Storage, Firestore, and Looker.

3. User interface and experience

Pluto AI’s intuitive interface makes AI tools accessible to users with varying technical expertise. A dropdown menu allows users to easily navigate between different modules and functionalities. The platform’s design prioritizes user experience, ensuring that employees can leverage AI capabilities without a steep learning curve.

Impact and adoption

Pluto AI has seen impressive adoption rates, with hundreds of daily active users across different departments. The platform’s user-friendly design and practical applications have garnered positive feedback from Magyar Telekom employees.

In addition, Pluto AI has enabled the development of various AI assistants, including legal and compliance assistants that accelerate contract review, identify compliance risks, and analyze legal documents. Knowledge management assistants have enhanced knowledge sharing and retrieval across the organization, while software development has benefited from code generation and assistance tools. Additionally, AI-powered chatbots that handle routine inquiries have significantly improved customer support experiences.

Magyar Telekom has seen quantifiable results since rolling out Pluto AI. These include hundreds of daily unique users, tens of thousands of API calls, an estimated 20% reduction in the time spent reviewing legal documents, and a 15% decrease in code defects.

Vision and future roadmap for Pluto AI

Magyar Telekom sees Pluto AI as a key part of its AI strategy going forward. To maximize its impact, the company intends to expand the platform to more markets, business units, and departments within the organization. Additionally, Magyar Telekom is looking into the possibility of offering Pluto AI as a service or a product to other Deutsche Telekom markets. The company is also planning to build a library of reusable AI modules and frameworks that can be easily adapted to different use cases.

Magyar Telekom is pursuing several key initiatives to enhance Pluto AI and expand its capabilities. These efforts include investigating the potential of agent-based AI systems to automate complex tasks and workflows, adding a language selector for multilingual support to cater to a diverse user base, and developing an enhanced interface for managing RAG solutions, monitoring usage, and tracking performance metrics. Magyar Telecom also plans to continue developing dashboards for monitoring and optimizing cloud resource usage and costs.

Pluto AI has transformed Magyar Telekom’s AI landscape, making AI accessible, practical, and impactful. By providing a user-friendly platform, fostering experimentation, and delivering tangible business value, Pluto AI has set a new standard for internal AI adoption.

Colab Enterprise is a collaborative, managed notebook environment with the security and compliance capabilities of Google Cloud. Powerful integrated AI, seamless collaboration tools, enterprise readiness, and zero-config flexible compute are some of the many features making Colab Enterprise a trusted tool for developers at companies of all sizes. 

Today, we’re excited to announce new productivity boosting capabilities in Colab Enterprise, including:

1. Code assistance powered by Gemini to improve code development

2. A Notebook gallery that helps you find sample notebooks to jumpstart your workflows

3. A UX redesign to improve the editor experience and asset organization

Gemini in Colab Enterprise

The latest version of Gemini, Google’s largest and most capable AI model, is now directly available in Colab Enterprise. With this integration, users can now use AI to assist with code completion and generation, increasing their productivity and decreasing time to value:

Code completion: With Code completion, customers can now start coding in their notebook and receive suggestions about potential ways to complete their code.

Code generation: With Code generation, customers can use Gemini to generate code for them based on a prompt.

Explain error: With Explain Error, customers can get an explanation of why errors occurred, giving information that’s helpful in debugging.

Fix error: With Fix Error, customers can ask for help to fix errors in your code so that you don’t have to consult external sources.

Sample Notebook Gallery

The Notebook gallery offers a one-stop shop to effortlessly discover, search, and build off of sample notebooks. These samples, code snippets, and getting started guides provide a practical, hands-on approach to learning new techniques, understanding best practices, and jumpstarting projects with ready-to-use templates and examples.

Notebook categories

• Notebooks in the gallery are organized by categories including “Getting Started”, “Partner Models”, and “RAG” making it easy to find relevant samples to build off of and accelerate your workflows. Use the dropdown arrows to explore notebooks within each category, and simply click to open.

Notebook tags and metadata

• See detailed information about sample notebooks before opening them, including a short description of the notebook’s contents, what modalities the notebook covers (e.g. text, image, video), and which AI models are used.

Search

• Use the gallery search bar to find sample notebooks using freeform text. Search based on keywords such as the name of the notebook or any of the listed metadata like type of model, and modality.

A UX refresh

We’ve redesigned Colab Enterprise to improve developer productivity. You can now access a new centralized dashboard to manage all your assets, an expanded editor for a more focused coding experience, a new dark mode, and integrations with other Vertex AI services such as Experiments, Model Evaluations, Tuning, Scheduler and Ray.

The new centralized dashboard which includes:

• Your private and shared notebooks

• Runtimes, templates, executions, and schedules

• A sample notebook gallery.

The new dark mode in Colab Enterprise boosts developer productivity by creating a more comfortable coding environment that minimizes eye fatigue during extended work periods.

We’ve also enhanced the core editor experience, which now includes:

• Expanded editor real estate, giving you more room to focus on what matters most: – writing code.

• A deeply integrated editor with MLOps tooling, so you can access your experiments, see model evaluation results, connect to Ray clusters, schedule a notebook run and much more, all accessible in a single MLOps panel.

• An easily accessible File menu system to find all the quick actions related to your notebook file and the editor.

• A stateful UI, so you can browse all your assets on the dashboard without losing all your open notebooks.

Get started today

Check these features out in Vertex AI Colab Enterprise today (console, documentation).

The AI era has supercharged expectations: users now issue more complex queries and demand pinpoint results, meaning there’s an 82% chance of losing a customer if they can’t quickly find what they need. Similarly, AI agents require ultra-relevant context for reliable task execution. However, when traditional search methods deliver noise – with generally up to 70% of retrieved passages lacking a true answer – both agentic workflows and user experiences suffer from untrustworthy and unreliable results.

To help businesses meet these rising expectations, we’re launching our new state-of-the-art Vertex AI Ranking API. It makes it easy to boost the precision of information surfaced within search, agentic workflows, and retrieval-augmented generation (RAG) systems. This means you can elevate your legacy search system and AI application in minutes, not months.

Go beyond simple retrieval

This is where precise ranking becomes essential. Think of the Vertex AI Ranking API as the precision filter at the crucial final stage of your retrieval pipeline. It intelligently sifts through the initial candidate set, identifying and elevating only the most pertinent information. This refinement step is key to unlocking higher quality, more trustworthy, and more efficient AI applications.

Vertex AI Ranking API acts as this powerful, yet easy-to-integrate, refinement layer. It takes the candidate list from your existing search or retrieval system and re-orders it based on deep semantic understanding, ensuring the best results rise to the top. Here’s how it helps you uplevel your systems:

1. Upgrade legacy search systems: Easily add state-of-the-art relevance scoring to existing search outputs, improving user satisfaction and business outcomes on commercial searches without overhauling your current stack.

2. Strengthen RAG systems: Send fewer, more relevant documents to your generative models. This improves answer trustworthiness while reducing latency and operating costs by optimizing context window usage.

3. Support intelligent agents: Guide AI agents with highly relevant information, streamlining their context and traces, and significantly improving the success rate of task completion.

What’s new in Ranking API

Today, we’re launching our new semantic reranker models:

• semantic-ranker-default-004 – our most accurate model for any use case
• semantic-ranker-fast-004 – our fastest model for latency-critical use cases

Our model establishing a new benchmark for ranking performance:

• State-of-the-art ranking: Based on evaluations using the industry-standard BEIR dataset, our model leads in accuracy among competitive standalone reranking API services. The nDCG is a metric that’s used to evaluate the quality of a ranking system by assessing how well ranked items align with their actual relevance and prioritizes relevant results at the top. We’ve published our evaluation scripts to ensure reproducibility of results.

• Industry-leading low latency: Our default model (semantic-ranker-default-004) is at least 2x faster than competitive reranking API services at any scale. Our fast model (semantic-ranker-fast-004) is tuned for latency-critical applications and typically exhibits 3x lower latency than our default model.

We’re also launching long context ranking with a limit of 200k total tokens per API request. Providing longer documents to the Ranking API allows it to better understand nuanced relationships between queries and information such as for customer reviews or product specifications in Retail. 

Real-world impact across domains

The benefits aren’t just theoretical. Benchmarks on industry-specific datasets demonstrate that integrating the Ranking API can significantly boost the quality of search results across diverse high-value domains such as retail, news, finance, and healthcare.

Elevate your search results in minutes

We designed the Vertex AI Ranking API for seamless integration. Adding this powerful relevance layer is straightforward, with several options:

• Try it live: Experience the difference on real-world data by enabling our Ranking API in the interactive Vertex Vector Search demo (link)  

• Build with Vertex AI: Integrate directly into any existing system for maximum flexibility (link)

• Enable it in RAG Engine: Select Ranking API in your RAG Engine to get more robust and accurate answers from your generative AI applications (link) 

• Use it in AlloyDB: For a truly streamlined experience, leverage the built-in ai.rank() SQL function directly within AlloyDB – a novel integration simplifying search use cases with AlloyDB (link)

• AI Frameworks: Use our native integrations with popular AI frameworks like GenKit and LangChain (link)

• Use it in Elasticsearch: Quickly boost accuracy with our built-in Ranking API integration in Elasticsearch (link)