[{"data":1,"prerenderedAt":1296},["ShallowReactive",2],{"/blog/best-practices-for-kubernetes-runners":3,"navigation-en-us":548,"banner-en-us":984,"footer-en-us":992,"blog-post-authors-en-us-Sara Kassabian":1237,"blog-related-posts-en-us-best-practices-for-kubernetes-runners":1251,"blog-share-wrapper-en-us":1276,"next-steps-en-us":1286},{"id":4,"title":5,"authors":6,"body":8,"category":528,"date":529,"description":530,"extension":531,"externalUrl":532,"faq":532,"featured":533,"heroImage":534,"hidePromo":533,"meta":535,"navigation":289,"path":536,"seo":537,"slug":541,"stem":542,"tags":543,"template":546,"updatedDate":532,"__hash__":547},"blogPosts/en-us/blog/best-practices-for-kubernetes-runners.md","Best practices to keep your Kubernetes runners moving",[7],"Sara Kassabian",{"type":9,"value":10,"toc":514},"minimark",[11,15,25,34,37,42,45,55,62,65,70,73,81,84,92,95,98,107,118,121,124,128,131,134,137,144,384,387,391,398,401,404,408,411,414,418,422,425,428,432,440,454,457,460,473,477,496,510],[12,13,14],"p",{},"Sometimes in software engineering, you have to learn the hard way. GitLab CI is extremely powerful and flexible, but it’s also easy to make mistakes that could take out a GitLab runner, which can clog up Sidekiq and bring down your entire GitLab instance.",[12,16,17,18,24],{},"Luckily, Sean Smith, senior software engineer for F5 Networks has been through it, and summarizes some of their learnings in ",[19,20,23],"a",{"href":21,"rel":22},"https://www.youtube.com/watch?v=Hks5ElUxkP4",[],"his talk at GitLab Commit San Francisco",". In the presentation, Sean goes in-depth about a past incident that clogged up F5 Network's GitLab runner, and shares tips on setting limits for Kubernetes (K8s) runners.",[12,26,27,28,33],{},"Sean is a GitLab administrator for ",[19,29,32],{"href":30,"rel":31},"https://www.f5.com/",[],"F5 Networks",", a company with about 1,800 users worldwide running 7,500 projects each month – excluding forks. That’s roughly 350,000 - 400,000 CI jobs going through the K8s runners each month. Until some recent hires, there were only three engineers to handle it all.",[12,35,36],{},"Instead of running a giant GitLab instance on one VM, F5 broke up their instance into seven different servers: Two HA web servers, one PostGres server, PostGres replica, Sidekiq, Gitaly (our Git filesystem), and Redis.",[38,39,41],"h2",{"id":40},"keep-your-gitlab-runners-up-and-moving","Keep your GitLab runners up and moving",[12,43,44],{},"F5 uses two types of GitLab runners:",[46,47,48,52],"ul",{},[49,50,51],"li",{},"Kubernetes: About 90% of F5 jobs go through K8s",[49,53,54],{},"Docker: Docker machine is run on-prem and in the cloud",[12,56,57,61],{},[58,59,60],"strong",{},"Why use Docker?"," F5 uses Docker to configure cluster networks in different jobs as well as for unit testing. Since the Docker machine can run on-prem and also in the cloud, it’s easy to have a VM dedicated to the job that allows you to manage those Docker images and Docker containers and set up your cluster networking topology within Docker, so you can run your tests and tear it down afterward without affecting other users. This isn’t something that is really possible in Kubernetes runners.",[12,63,64],{},"Otherwise, F5 Networks uses Kubernetes, but keeping your K8s up and running isn’t necessarily foolproof.",[66,67,69],"h3",{"id":68},"ci-jobs-can-spawn","CI jobs can spawn",[12,71,72],{},"Sometimes, a seemingly benign coding error can create unanticipated consequences for your Kubernetes runners.",[12,74,75,76,80],{},"One time, an F5 Engineer decided to use a GitLab CI job to automatically configure different settings on various jobs and projects. It made sense to configure using GitLab CI because the engineer wanted to be able to use ",[19,77,79],{"href":78},"/topics/version-control/","Git for version control",". Version control makes it easier for the team to iterate on the code transparently. He wrote the code to run the job.",[12,82,83],{},"But, he didn’t read the fine print in the library he was using. The code he wrote looked for the project ID, and if it found the project ID, runs the pipeline once per hour at the 30-minute mark. The assumption was that if there was already a matching scheduled task, the create function would not create a duplicate. Unfortunately, this was not the case. The code he ran caused the number of CI jobs to grow exponentially.",[12,85,86,91],{},[87,88],"img",{"alt":89,"src":90},"The code that clogged the K8s runner with GitLab CI jobs for F5 Networks","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398862/blog/Content%20Images/problemcode.png","\nThe code that clogged the K8s runner with GitLab CI jobs for F5 Networks. Can you see the problem yet?",[12,93,94],{},"\"You schedule a job, then next you schedule another job so now you've got two jobs scheduled, and then you've got four jobs scheduled, and then eight, after 10 iterations, you get around the 1,024 jobs scheduled and after 1,532,000 jobs, if this was allowed to run for 24 hours, you would end up with 16.7 million jobs being scheduled by the 24th hour,\" says Sean.",[12,96,97],{},"In short: Chaos. Remember, F5 Networks has a CI pipeline capacity of 350,000 to 400,000 jobs per month, so 16.7 million jobs in 24 hours could easily clog the system, taking down the K8s nodes, as well as GitLab nodes.",[12,99,100,101,106],{},"Luckily, there’s a simple enough fix. First, identify which project is causing the problem, and disable CI on the project so it can’t create any new jobs. Next, kill all the pending jobs by ",[19,102,105],{"href":103,"rel":104},"https://gitlab.com/snippets/1924269",[],"running this snippet",".",[108,109,115],"pre",{"className":110,"code":112,"language":113,"meta":114},[111],"language-text","# gitlab-rails console\np = Project.find_by_full_path(‘rogue-group/rogue-project’)\nCi::Pipeline.where(project_id: p.id).where(status: ‘pending’).each {|p| p.cancel}\nexit\n","text","",[116,117,112],"code",{"__ignoreMap":114},[12,119,120],{},"It’s really a judgment call whether to kill a running job or not. If a job is currently running and is going to take all of 30 seconds then maybe don’t bother killing it, but if the job is going to take 30 minutes then consider killing it to free up resources for your users.",[12,122,123],{},"F5 learned a lesson here and set up a monitoring alert to help ensure the job queue doesn’t back up like that again. The Cron job checks to make sure F5 is not exceeding a preestablished threshold on the number of jobs in a pending state. The alert links to a dashboard and also includes the full playbook for how to resolve the problem (because let’s face it, nobody is at their best when troubleshooting bleary-eyed at 3 a.m.). At first there were some false positives, but now the alerting has been fine-tuned and the system saved F5 from two outages so far.",[66,125,127],{"id":126},"push-it-to-the-limit","Push it to the limit",[12,129,130],{},"The fact is, nobody has an unlimited cloud budget, and even if you're on-prem, resources are even more constrained for users that rely upon hardware. Sean says that F5 soon realized that, to meet the needs of all users, sensible limits had to be established so one or two mega-users didn't devour all their resources. He has some tips on how to set limits in your Kubernetes and GitLab runners.",[12,132,133],{},"While some users may be disgruntled that cloud limits exist and are enforced, the best method is to keep an open dialogue with users about the limits while recognizing that projects expand and grow over a period of time.",[12,135,136],{},"Fortunately you can set the limits yourself and don’t have to rely on the goodwill of your users to conserve CPU. Kubernetes allows limits by default, and GitLab supports K8s request and limits. The K8s scheduler uses requests to determine which nodes to run the workload on. Limits will kill a job if the job exceeds the predefined limit – there can be different requests and limits but if requests aren’t specified and limits are, the scheduler will use the limits to determine the request value.",[12,138,139,106],{},[19,140,143],{"href":141,"rel":142},"https://gitlab.com/snippets/1926912",[],"Take a peek at what F5 configured the limits for their Kubernetes GitLab runner",[108,145,149],{"className":146,"code":147,"language":148,"meta":114,"style":114},"language-ruby shiki shiki-themes github-light","concurrent = 200\nlog_format = \"json\"\n[[runners]]\n  name = \"Kubernetes Gitlab Runner\"\n  url = \"https://gitlab.example.com/ci\"\n  token = \"insert token here\"\n  executor = \"kubernetes\"\n  [runners.kubernetes]\n    namespace = \"gitlab-runner\"\n    service-account = \"gitlab-runner-user\"\n    pull_policy = \"always\"\n\n    # build container\n    cpu_limit = \"2\"\n    memory_limit = \"6Gi\"\n\n    # service containers\n    service_cpu_limit = \"1\"\n    service_memory_limit = \"1Gi\"\n\n    # helper container\n    helper_cpu_limit = \"1\"\n    helper_memory_limit = \"1Gi\"\n\n","ruby",[116,150,151,168,180,187,198,209,220,231,244,255,273,284,291,298,309,320,325,331,342,353,358,364,374],{"__ignoreMap":114},[152,153,156,160,164],"span",{"class":154,"line":155},"line",1,[152,157,159],{"class":158},"sqxcx","concurrent",[152,161,163],{"class":162},"sD7c4"," =",[152,165,167],{"class":166},"sYu0t"," 200\n",[152,169,171,174,176],{"class":154,"line":170},2,[152,172,173],{"class":158},"log_format",[152,175,163],{"class":162},[152,177,179],{"class":178},"sYBdl"," \"json\"\n",[152,181,183],{"class":154,"line":182},3,[152,184,186],{"class":185},"sgsFI","[[runners]]\n",[152,188,190,193,195],{"class":154,"line":189},4,[152,191,192],{"class":158},"  name",[152,194,163],{"class":162},[152,196,197],{"class":178}," \"Kubernetes Gitlab Runner\"\n",[152,199,201,204,206],{"class":154,"line":200},5,[152,202,203],{"class":158},"  url",[152,205,163],{"class":162},[152,207,208],{"class":178}," \"https://gitlab.example.com/ci\"\n",[152,210,212,215,217],{"class":154,"line":211},6,[152,213,214],{"class":158},"  token",[152,216,163],{"class":162},[152,218,219],{"class":178}," \"insert token here\"\n",[152,221,223,226,228],{"class":154,"line":222},7,[152,224,225],{"class":158},"  executor",[152,227,163],{"class":162},[152,229,230],{"class":178}," \"kubernetes\"\n",[152,232,234,237,241],{"class":154,"line":233},8,[152,235,236],{"class":185},"  [runners.",[152,238,240],{"class":239},"s7eDp","kubernetes",[152,242,243],{"class":185},"]\n",[152,245,247,250,252],{"class":154,"line":246},9,[152,248,249],{"class":158},"    namespace",[152,251,163],{"class":162},[152,253,254],{"class":178}," \"gitlab-runner\"\n",[152,256,258,261,264,267,270],{"class":154,"line":257},10,[152,259,260],{"class":185},"    service",[152,262,263],{"class":162},"-",[152,265,266],{"class":185},"account ",[152,268,269],{"class":162},"=",[152,271,272],{"class":178}," \"gitlab-runner-user\"\n",[152,274,276,279,281],{"class":154,"line":275},11,[152,277,278],{"class":158},"    pull_policy",[152,280,163],{"class":162},[152,282,283],{"class":178}," \"always\"\n",[152,285,287],{"class":154,"line":286},12,[152,288,290],{"emptyLinePlaceholder":289},true,"\n",[152,292,294],{"class":154,"line":293},13,[152,295,297],{"class":296},"sAwPA","    # build container\n",[152,299,301,304,306],{"class":154,"line":300},14,[152,302,303],{"class":158},"    cpu_limit",[152,305,163],{"class":162},[152,307,308],{"class":178}," \"2\"\n",[152,310,312,315,317],{"class":154,"line":311},15,[152,313,314],{"class":158},"    memory_limit",[152,316,163],{"class":162},[152,318,319],{"class":178}," \"6Gi\"\n",[152,321,323],{"class":154,"line":322},16,[152,324,290],{"emptyLinePlaceholder":289},[152,326,328],{"class":154,"line":327},17,[152,329,330],{"class":296},"    # service containers\n",[152,332,334,337,339],{"class":154,"line":333},18,[152,335,336],{"class":158},"    service_cpu_limit",[152,338,163],{"class":162},[152,340,341],{"class":178}," \"1\"\n",[152,343,345,348,350],{"class":154,"line":344},19,[152,346,347],{"class":158},"    service_memory_limit",[152,349,163],{"class":162},[152,351,352],{"class":178}," \"1Gi\"\n",[152,354,356],{"class":154,"line":355},20,[152,357,290],{"emptyLinePlaceholder":289},[152,359,361],{"class":154,"line":360},21,[152,362,363],{"class":296},"    # helper container\n",[152,365,367,370,372],{"class":154,"line":366},22,[152,368,369],{"class":158},"    helper_cpu_limit",[152,371,163],{"class":162},[152,373,341],{"class":178},[152,375,377,380,382],{"class":154,"line":376},23,[152,378,379],{"class":158},"    helper_memory_limit",[152,381,163],{"class":162},[152,383,352],{"class":178},[12,385,386],{},"\"We have got currency of 200 jobs, so it will at max spawn 200 jobs and you'll see that we are limiting the CPU use on the build container to two and memory to six gigabytes, and on the helper and service CPU and memory limits, we have one CPU and one gig of memory each,\" says Sean. \"And so it gives you that flexibility to break it out because generally, you don't necessarily need as much CPU or as much memory on a service that you're spending up in your CI job.\"",[38,388,390],{"id":389},"what-comes-first-setting-up-kubernetes-runners-or-establishing-limits","What comes first: Setting up Kubernetes runners or establishing limits?",[12,392,393,397],{},[19,394,396],{"href":395},"/topics/devops/","DevOps"," is a data-driven practice, so the idea of setting limits to conserve resources without any underlying data about what users are doing can seem counterintuitive. If you’re migrating to Kubernetes runners from a Docker runner or a shell runner, it’s easy enough to extrapolate the numbers to establish limits as you set up your Kuberntes runners.",[12,399,400],{},"If you’re brand-new to GitLab and GitLab CI, then it’s kind of a shot in the dark. Think about your bills and resource constraints: How much memory and CPU is available? Is anything else running on your K8s cluster. Chances are, your guesses will be incorrect – but that’s OK.",[12,402,403],{},"It might sound obvious, but if you’re running a hosted application on the same K8s cluster as your GitLab CI jobs, don’t set limits based on the capacity of a full K8s cluster. Ideally, you’d have a separate K8s cluster for GitLab CI jobs, but that isn’t always possible.",[66,405,407],{"id":406},"how-f5-networks-did-it","How F5 Networks did it",[12,409,410],{},"F5 Networks started with a small team of roughly 50 people and maybe 100 projects in GitLab – so setting a limit on K8s wasn’t a major concern until the company and, as a result, projects, started to grow.",[12,412,413],{},"Once it came time to set limits to their preexisting K8s runners, the first step was to enable the K8s metric server to monitor how their users consume resources. The next step was to determine what users are doing. Sean recommends using a tool like Grafana or Prometheus, which has a native integration within GitLab (although, F5 used a tool called K9), to extract the data from the K8s metric server and display it on some sort of dashboard using Grafana or Prometheus.",[38,415,417],{"id":416},"some-more-tips-for-kubernetes-runners","Some more tips for Kubernetes runners",[66,419,421],{"id":420},"cutting-them-off-enforcing-limits","Cutting them off: Enforcing limits",[12,423,424],{},"Once a user hits their limit, most of the time the end result is their job gets killed. Usually the user will notice a mistake, go in, and fix their code, but most likely they will just ask for more resources.",[12,426,427],{},"The best way to determine whether or not to allocate more of your finite resources to a user is to determine need, Sean explains. Ask the user to return to you with concrete numbers about the amount of RAM or CPU they require. But if you don’t have the resources, then don’t overextend yourselves to the detriment of your other users.",[66,429,431],{"id":430},"use-labels-to-reveal-more-data","Use labels to reveal more data",[12,433,434,439],{},[19,435,438],{"href":436,"rel":437},"https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set",[],"Labels"," make it easier to identify workloads in Kubernetes, and can be expanded to environmental variables within GitLab, for example, job = \"$CI_JOB_ID\" and project = \"$CI_PROJECT_ID\". Labels can be used by admins who are manually doing Quebectal commands against K8s or they can be used in reporting tools like Prometheus or Grafana for setting limits. But labels are the most valuable when it comes to debugging purposes.",[12,441,442,443,448,449,453],{},"Bear in mind, labels are finicky in Kubernetes. ",[19,444,447],{"href":445,"rel":446},"https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4565",[],"There are certain characters (stay away from \"?\") that can cause jobs to fail",". There is a 63 character limit on labels. If there is an unsupported character or the label is too long, the job won’t start. There won’t be a really good indication as to why your job wouldn’t start either, which can be a pain for troubleshooting. ",[19,450,452],{"href":436,"rel":451},[],"Bookmark this page to learn more about labels in Kubernetes"," (including its limitations).",[12,455,456],{},"GitLab users that run on K8s need to be cautious not to overburden the runner with GitLab CI jobs, and ought to consider setting limits on CPU to conserve valuable resources.",[12,458,459],{},"Want to learn more about how F5 manages their Kubernetes runners on their GitLab instance? Watch Sean's presentation at GitLab Commit San Francisco in the video below.",[461,462,465,466],"figure",{"className":463},[464],"video_container","\n  ",[467,468,472],"iframe",{"src":469,"frameBorder":470,"allowFullScreen":471},"https://www.youtube-nocookie.com/embed/Hks5ElUxkP4","0","true"," ",[38,474,476],{"id":475},"learn-more","Learn more",[46,478,479,487],{},[49,480,481,486],{},[19,482,485],{"href":483,"rel":484},"https://docs.gitlab.com/user/workspace/set_up_gitlab_agent_and_proxies/",[],"Read on"," to learn more about how GitLab and Kubernetes work together, and explore our plans for future integration with Kubernetes.",[49,488,489,490,495],{},"Explore the official documentation on ",[19,491,494],{"href":492,"rel":493},"https://docs.gitlab.com/runner/executors/kubernetes/",[],"Kubernetes executor",", which covers everything from choosing options in your configuration file to giving GitLab Runner access to the Kubernetes API, environment variables, volumes, helper containers, security context, privileged mode, secret volume, and removing old runner pods.",[12,497,498,499,504,505],{},"Cover Photo by ",[19,500,503],{"href":501,"rel":502},"https://unsplash.com/@rockthechaos?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText",[],"Kolleen Gladden"," on ",[19,506,509],{"href":507,"rel":508},"https://unsplash.com/s/photos/track-and-field?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText",[],"Unsplash",[511,512,513],"style",{},"html pre.shiki code .sqxcx, html code.shiki .sqxcx{--shiki-default:#E36209}html pre.shiki code .sD7c4, html code.shiki .sD7c4{--shiki-default:#D73A49}html pre.shiki code .sYu0t, html code.shiki .sYu0t{--shiki-default:#005CC5}html pre.shiki code .sYBdl, html code.shiki .sYBdl{--shiki-default:#032F62}html pre.shiki code .sgsFI, html code.shiki .sgsFI{--shiki-default:#24292E}html pre.shiki code .s7eDp, html code.shiki .s7eDp{--shiki-default:#6F42C1}html pre.shiki code .sAwPA, html code.shiki .sAwPA{--shiki-default:#6A737D}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":114,"searchDepth":170,"depth":170,"links":515},[516,520,523,527],{"id":40,"depth":170,"text":41,"children":517},[518,519],{"id":68,"depth":182,"text":69},{"id":126,"depth":182,"text":127},{"id":389,"depth":170,"text":390,"children":521},[522],{"id":406,"depth":182,"text":407},{"id":416,"depth":170,"text":417,"children":524},[525,526],{"id":420,"depth":182,"text":421},{"id":430,"depth":182,"text":431},{"id":475,"depth":170,"text":476},"engineering","2020-05-27","In a presentation at GitLab Commit San Francisco, a senior software engineer from F5 Networks shares some best practices for working with Kubernetes runners.","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749681341/Blog/Hero%20Images/trackandfield.jpg",{},"/en-us/blog/best-practices-for-kubernetes-runners",{"title":5,"description":530,"ogTitle":5,"ogDescription":530,"noIndex":533,"ogImage":534,"ogUrl":538,"ogSiteName":539,"ogType":540,"canonicalUrls":538},"https://about.gitlab.com/blog/best-practices-for-kubernetes-runners","https://about.gitlab.com","article","best-practices-for-kubernetes-runners","en-us/blog/best-practices-for-kubernetes-runners",[240,544,545],"CI/CD","user stories","BlogPost","dIZPhtDdxYkbRmiNXf2hSWj2y3tPufffrZRr9v0hziw",{"logo":549,"freeTrial":554,"sales":559,"login":564,"items":569,"search":903,"minimal":935,"duo":954,"switchNav":963,"pricingDeployment":974},{"config":550},{"href":551,"dataGaName":552,"dataGaLocation":553},"/","gitlab logo","header",{"text":555,"config":556},"Get free trial",{"href":557,"dataGaName":558,"dataGaLocation":553},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":560,"config":561},"Request a demo",{"href":562,"dataGaName":563,"dataGaLocation":553},"/request-a-demo/","sales",{"text":565,"config":566},"Sign in",{"href":567,"dataGaName":568,"dataGaLocation":553},"https://gitlab.com/users/sign_in/","sign in",[570,654,742,747,827,881],{"text":571,"left":289,"config":572,"menu":574},"Platform",{"dataNavLevelOne":573},"platform",{"type":575,"link":576,"columns":580,"feature":643},"lists",{"text":577,"config":578},"Explore the Platform",{"href":579,"dataGaName":573,"dataGaLocation":553},"/platform/",[581,602,625],{"title":396,"items":582},[583,587,592,597],{"text":544,"config":584},{"href":585,"dataGaLocation":553,"dataGaName":586},"/solutions/continuous-integration/","ci/cd",{"text":588,"config":589},"Source Code Management",{"href":590,"dataGaLocation":553,"dataGaName":591},"/solutions/source-code-management/","source code management",{"text":593,"config":594},"Agile Delivery",{"href":595,"dataGaLocation":553,"dataGaName":596},"/solutions/agile-delivery/","agile delivery",{"text":598,"config":599},"Artifact Management",{"href":600,"dataGaLocation":553,"dataGaName":601},"/artifact-central/","artifact management",{"title":603,"items":604},"Security & Governance",[605,610,615,620],{"text":606,"config":607},"Application Security Testing",{"href":608,"dataGaLocation":553,"dataGaName":609},"/solutions/application-security-testing/","application security testing",{"text":611,"config":612},"Governance & Compliance",{"href":613,"dataGaLocation":553,"dataGaName":614},"/solutions/software-compliance/","governance and compliance",{"text":616,"config":617},"Secrets Management",{"href":618,"dataGaLocation":553,"dataGaName":619},"/gitlab-secrets-manager/","secrets management",{"text":621,"config":622},"Supply Chain Security",{"href":623,"dataGaLocation":553,"dataGaName":624},"/solutions/supply-chain/","supply chain security",{"title":626,"items":627},"Agentic AI & Context",[628,633,638],{"text":629,"config":630},"Duo Agent Platform",{"href":631,"dataGaLocation":553,"dataGaName":632},"/gitlab-duo-agent-platform/","duo agent platform",{"text":634,"config":635},"Orbit Context Graph",{"href":636,"dataGaLocation":553,"dataGaName":637},"/gitlab-orbit/","orbit",{"text":639,"config":640},"Visibility & Measurement",{"href":641,"dataGaLocation":553,"dataGaName":642},"/solutions/visibility-measurement/","visibility and measurement",{"config":644,"title":647,"text":648,"link":649},{"background":645,"textColor":646},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1790185102/o7dn32anqcb6z8dgd3as.jpg')","#000","GitLab Orbit is generally available","Higher agent accuracy, lower token cost",{"text":650,"config":651},"Read the announcement",{"href":652,"dataGaName":653,"dataGaLocation":553},"/blog/gitlab-orbit-ga/","orbit ga announcement",{"text":655,"left":289,"config":656,"menu":658},"Solutions",{"dataNavLevelOne":657},"solutions",{"type":575,"link":659,"columns":663,"feature":733},{"text":660,"config":661},"View all Solutions",{"href":662,"dataGaName":657,"dataGaLocation":553},"/solutions/",[664,682,700],{"title":665,"items":666},"Outcomes",[667,672,677],{"text":668,"config":669},"DevOps Modernization",{"href":670,"dataGaLocation":553,"dataGaName":671},"/assessments/devops-modernization-assessment/","devops modernization",{"text":673,"config":674},"Security Modernization",{"href":675,"dataGaLocation":553,"dataGaName":676},"/assessments/security-modernization-assessment/","security modernization",{"text":678,"config":679},"AI Modernization",{"href":680,"dataGaLocation":553,"dataGaName":681},"/assessments/ai-modernization-assessment/","ai modernization",{"title":683,"items":684},"By size",[685,690,695],{"text":686,"config":687},"Enterprise",{"href":688,"dataGaLocation":553,"dataGaName":689},"/enterprise/","enterprise",{"text":691,"config":692},"Small Business",{"href":693,"dataGaLocation":553,"dataGaName":694},"/small-business/","small business",{"text":696,"config":697},"Startups",{"href":698,"dataGaLocation":553,"dataGaName":699},"/solutions/startups/","startups",{"title":701,"items":702},"Industries",[703,708,713,718,723,728],{"text":704,"config":705},"Financial Services",{"href":706,"dataGaLocation":553,"dataGaName":707},"/solutions/finance/","financial services",{"text":709,"config":710},"Public Sector",{"href":711,"dataGaLocation":553,"dataGaName":712},"/solutions/public-sector/","public sector",{"text":714,"config":715},"Telecommunications",{"href":716,"dataGaLocation":553,"dataGaName":717},"/solutions/telecommunications/","telecommunications",{"text":719,"config":720},"Automotive",{"href":721,"dataGaLocation":553,"dataGaName":722},"/solutions/automotive/","automotive",{"text":724,"config":725},"Education",{"href":726,"dataGaLocation":553,"dataGaName":727},"/solutions/education/","education",{"text":729,"config":730},"Aerospace",{"href":731,"dataGaLocation":553,"dataGaName":732},"/solutions/aerospace/","aerospace",{"config":734,"title":735,"text":736,"link":737},{"background":645,"textColor":646},"GitLab Security Standard","Learn how to harden your software factory and let AI agents earn autonomy, without trading away security.",{"text":738,"config":739},"Explore the standard",{"href":740,"dataGaName":741,"dataGaLocation":553},"/gitlab-security-standard/","gitlab security standard",{"text":743,"config":744},"Pricing",{"href":745,"dataGaName":746,"dataGaLocation":553,"dataNavLevelOne":746},"/pricing/","pricing",{"text":748,"config":749,"menu":751},"Resources",{"dataNavLevelOne":750},"resources",{"type":575,"link":752,"columns":756,"feature":818},{"text":753,"config":754},"View all resources",{"href":755,"dataGaName":750,"dataGaLocation":553},"/resources/",[757,790],{"title":758,"items":759},"Discover",[760,765,770,775,780,785],{"text":761,"config":762},"Docs",{"href":763,"dataGaName":764,"dataGaLocation":553},"https://docs.gitlab.com/","docs",{"text":766,"config":767},"University",{"href":768,"dataGaName":769,"dataGaLocation":553},"https://university.gitlab.com/","university",{"text":771,"config":772},"Demo Series",{"href":773,"dataGaName":774,"dataGaLocation":553},"/gitlab-technical-demo-series/","demo series",{"text":776,"config":777},"Demo Hub",{"href":778,"dataGaName":779,"dataGaLocation":553},"/demo-hub/","demo hub",{"text":781,"config":782},"Services",{"href":783,"dataGaName":784,"dataGaLocation":553},"/services/","services",{"text":786,"config":787},"Well Architected",{"href":788,"dataGaName":789,"dataGaLocation":553},"/resources/well-architected/","well architected",{"title":791,"items":792},"Connect",[793,798,803,808,813],{"text":794,"config":795},"Blog",{"href":796,"dataGaName":797,"dataGaLocation":553},"/blog/","blog",{"text":799,"config":800},"Community",{"href":801,"dataGaName":802,"dataGaLocation":553},"/community/","community",{"text":804,"config":805},"Customers",{"href":806,"dataGaName":807,"dataGaLocation":553},"/customers/","customers",{"text":809,"config":810},"Partners",{"href":811,"dataGaName":812,"dataGaLocation":553},"/partners/","partners",{"text":814,"config":815},"Events",{"href":816,"dataGaName":817,"dataGaLocation":553},"/events/","events",{"config":819,"title":820,"text":821,"link":822},{"background":645,"textColor":646},"What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":823,"config":824},"Read the latest",{"href":825,"dataGaName":826,"dataGaLocation":553},"/whats-new/","whats new",{"text":828,"config":829,"menu":831},"Company",{"dataNavLevelOne":830},"company",{"type":575,"columns":832},[833],{"items":834},[835,840,846,851,856,861,866,871,876],{"text":836,"config":837},"About",{"href":838,"dataGaName":839,"dataGaLocation":553},"/company/","about",{"text":841,"config":842,"footerGa":845},"Jobs",{"href":843,"dataGaName":844,"dataGaLocation":553},"/jobs/","jobs",{"dataGaName":844},{"text":847,"config":848},"Press",{"href":849,"dataGaName":850,"dataGaLocation":553},"/press/","press",{"text":852,"config":853},"Handbook",{"href":854,"dataGaName":855,"dataGaLocation":553},"https://handbook.gitlab.com/","handbook",{"text":857,"config":858},"Leadership",{"href":859,"dataGaName":860,"dataGaLocation":553},"/company/team/e-group/","leadership",{"text":862,"config":863},"Investor relations",{"href":864,"dataGaName":865,"dataGaLocation":553},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":867,"config":868},"Trust Center",{"href":869,"dataGaName":870,"dataGaLocation":553},"/security/","trust center",{"text":872,"config":873},"AI Transparency Center",{"href":874,"dataGaName":875,"dataGaLocation":553},"/ai-transparency-center/","ai transparency center",{"text":877,"config":878},"Newsletter",{"href":879,"dataGaName":880,"dataGaLocation":553},"/company/contact/#contact-forms","newsletter",{"text":882,"config":883,"menu":884},"Contact us",{"dataNavLevelOne":830},{"type":575,"columns":885},[886],{"items":887},[888,893,898],{"text":889,"config":890},"Talk to sales",{"href":891,"dataGaName":892,"dataGaLocation":553},"/sales/","talk to sales",{"text":894,"config":895},"Support portal",{"href":896,"dataGaName":897,"dataGaLocation":553},"https://support.gitlab.com/hc/en-us","support portal",{"text":899,"config":900},"Customer portal",{"href":901,"dataGaName":902,"dataGaLocation":553},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":904,"login":905,"suggestions":912},"Close",{"text":906,"link":907},"To search repositories and projects, login to",{"text":908,"config":909},"gitlab.com",{"href":567,"dataGaName":910,"dataGaLocation":911},"search login","search",{"text":913,"default":914},"Suggestions",[915,918,921,923,927,931],{"text":916,"config":917},"GitLab Duo Agent Platform",{"href":631,"dataGaName":916,"dataGaLocation":911},{"text":919,"config":920},"Code Suggestions (AI)",{"href":631,"dataGaName":919,"dataGaLocation":911},{"text":544,"config":922},{"href":585,"dataGaName":544,"dataGaLocation":911},{"text":924,"config":925},"GitLab on AWS",{"href":926,"dataGaName":924,"dataGaLocation":911},"/partners/technology-partners/aws/",{"text":928,"config":929},"GitLab on Google Cloud",{"href":930,"dataGaName":928,"dataGaLocation":911},"/partners/technology-partners/google-cloud-platform/",{"text":932,"config":933},"Why GitLab?",{"href":934,"dataGaName":932,"dataGaLocation":911},"/why-gitlab/",{"freeTrial":936,"mobileIcon":941,"desktopIcon":946,"secondaryButton":949},{"text":937,"config":938},"Start free trial",{"href":939,"dataGaName":558,"dataGaLocation":940},"https://gitlab.com/-/trials/new/","nav",{"altText":942,"config":943},"Gitlab Icon",{"src":944,"dataGaName":945,"dataGaLocation":940},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":942,"config":947},{"src":948,"dataGaName":945,"dataGaLocation":940},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":950,"config":951},"Get Started",{"href":952,"dataGaName":953,"dataGaLocation":940},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":955,"mobileIcon":959,"desktopIcon":961},{"text":956,"config":957},"Learn more about GitLab Duo",{"href":631,"dataGaName":958,"dataGaLocation":940},"gitlab duo",{"altText":942,"config":960},{"src":944,"dataGaName":945,"dataGaLocation":940},{"altText":942,"config":962},{"src":948,"dataGaName":945,"dataGaLocation":940},{"button":964,"mobileIcon":969,"desktopIcon":971},{"text":965,"config":966},"/switch",{"href":967,"dataGaName":968,"dataGaLocation":940},"#contact","switch",{"altText":942,"config":970},{"src":944,"dataGaName":945,"dataGaLocation":940},{"altText":942,"config":972},{"src":973,"dataGaName":945,"dataGaLocation":940},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":975,"mobileIcon":980,"desktopIcon":982},{"text":976,"config":977},"Back to pricing",{"href":745,"dataGaName":978,"dataGaLocation":940,"icon":979},"back to pricing","GoBack",{"altText":942,"config":981},{"src":944,"dataGaName":945,"dataGaLocation":940},{"altText":942,"config":983},{"src":948,"dataGaName":945,"dataGaLocation":940},{"title":985,"button":986,"config":990},"GitLab Orbit is generally available. Increase agent accuracy and lower token cost",{"text":650,"config":987},{"href":652,"dataGaName":988,"dataGaLocation":989},"orbit-ga-banner","global-banner",{"layout":991,"disabled":533},"release",{"data":993},{"text":994,"config":995,"items":1000,"minimal":1226},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"twitter":996,"facebook":997,"youtube":998,"linkedin":999},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[1001,1049,1106,1153,1194],{"title":743,"links":1002,"subMenu":1018},[1003,1008,1013],{"text":1004,"config":1005},"View plans",{"href":745,"dataGaName":1006,"dataGaLocation":1007},"view plans","footer",{"text":1009,"config":1010},"Why Premium?",{"href":1011,"dataGaName":1012,"dataGaLocation":1007},"/pricing/premium/","why premium",{"text":1014,"config":1015},"Why Ultimate?",{"href":1016,"dataGaName":1017,"dataGaLocation":1007},"/pricing/ultimate/","why ultimate",[1019],{"title":1020,"links":1021},"Contact Us",[1022,1025,1027,1029,1034,1039,1044],{"text":1023,"config":1024},"Contact sales",{"href":891,"dataGaName":563,"dataGaLocation":1007},{"text":894,"config":1026},{"href":896,"dataGaName":897,"dataGaLocation":1007},{"text":899,"config":1028},{"href":901,"dataGaName":902,"dataGaLocation":1007},{"text":1030,"config":1031},"Status",{"href":1032,"dataGaName":1033,"dataGaLocation":1007},"https://status.gitlab.com/","status",{"text":1035,"config":1036},"Terms of use",{"href":1037,"dataGaName":1038,"dataGaLocation":1007},"/terms/","terms of use",{"text":1040,"config":1041},"Privacy statement",{"href":1042,"dataGaName":1043,"dataGaLocation":1007},"/privacy/","privacy statement",{"text":1045,"config":1046},"Cookie preferences",{"dataGaName":1047,"dataGaLocation":1007,"id":1048,"isOneTrustButton":289},"cookie preferences","ot-sdk-btn",{"title":571,"links":1050,"subMenu":1066},[1051,1054,1056,1058,1060,1062],{"text":1052,"config":1053},"Explore the platform",{"href":579,"dataGaName":573,"dataGaLocation":1007},{"text":629,"config":1055},{"href":631,"dataGaName":632,"dataGaLocation":1007},{"text":544,"config":1057},{"href":585,"dataGaName":586,"dataGaLocation":1007},{"text":634,"config":1059},{"href":636,"dataGaName":637,"dataGaLocation":1007},{"text":616,"config":1061},{"href":618,"dataGaName":619,"dataGaLocation":1007},{"text":1063,"config":1064},"Why GitLab",{"href":934,"dataGaName":1065,"dataGaLocation":1007},"why gitlab",[1067],{"title":1068,"links":1069},"Topics",[1070,1074,1079,1082,1086,1091,1096,1101],{"text":544,"config":1071},{"href":1072,"dataGaName":1073,"dataGaLocation":1007},"/topics/ci-cd/","cicd",{"text":1075,"config":1076},"GitOps",{"href":1077,"dataGaName":1078,"dataGaLocation":1007},"/topics/gitops/","gitops",{"text":396,"config":1080},{"href":395,"dataGaName":1081,"dataGaLocation":1007},"devops",{"text":1083,"config":1084},"Version Control",{"href":78,"dataGaName":1085,"dataGaLocation":1007},"version control",{"text":1087,"config":1088},"DevSecOps",{"href":1089,"dataGaName":1090,"dataGaLocation":1007},"/topics/devsecops/","devsecops",{"text":1092,"config":1093},"Cloud Native",{"href":1094,"dataGaName":1095,"dataGaLocation":1007},"/topics/cloud-native/","cloud native",{"text":1097,"config":1098},"AI for Coding",{"href":1099,"dataGaName":1100,"dataGaLocation":1007},"/topics/devops/ai-for-coding/","ai for coding",{"text":1102,"config":1103},"Agentic AI",{"href":1104,"dataGaName":1105,"dataGaLocation":1007},"/topics/agentic-ai/","agentic ai",{"title":655,"links":1107},[1108,1111,1114,1117,1120,1123,1125,1127,1129,1131,1133,1136,1138,1141,1145,1149],{"text":1109,"config":1110},"DevOps modernization",{"href":670,"dataGaName":671,"dataGaLocation":1007},{"text":1112,"config":1113},"Security modernization",{"href":675,"dataGaName":676,"dataGaLocation":1007},{"text":1115,"config":1116},"AI modernization",{"href":680,"dataGaName":681,"dataGaLocation":1007},{"text":1118,"config":1119},"Financial services",{"href":706,"dataGaName":707,"dataGaLocation":1007},{"text":1121,"config":1122},"Public sector",{"href":711,"dataGaName":712,"dataGaLocation":1007},{"text":714,"config":1124},{"href":716,"dataGaName":717,"dataGaLocation":1007},{"text":719,"config":1126},{"href":721,"dataGaName":722,"dataGaLocation":1007},{"text":724,"config":1128},{"href":726,"dataGaName":727,"dataGaLocation":1007},{"text":729,"config":1130},{"href":731,"dataGaName":732,"dataGaLocation":1007},{"text":686,"config":1132},{"href":688,"dataGaName":689,"dataGaLocation":1007},{"text":1134,"config":1135},"Small business",{"href":693,"dataGaName":694,"dataGaLocation":1007},{"text":696,"config":1137},{"href":698,"dataGaName":699,"dataGaLocation":1007},{"text":1075,"config":1139},{"href":1140,"dataGaName":1078,"dataGaLocation":1007},"/solutions/gitops/",{"text":1142,"config":1143},"Software composition analysis",{"href":623,"dataGaName":1144,"dataGaLocation":1007},"software composition analysis",{"text":1146,"config":1147},"Value stream management",{"href":641,"dataGaName":1148,"dataGaLocation":1007},"value stream management",{"text":1150,"config":1151},"All solutions",{"href":662,"dataGaName":1152,"dataGaLocation":1007},"all solutions",{"title":748,"links":1154},[1155,1160,1165,1167,1169,1171,1173,1175,1177,1179,1181,1183,1185,1187,1190],{"text":1156,"config":1157},"Install",{"href":1158,"dataGaName":1159,"dataGaLocation":1007},"/install/","install",{"text":1161,"config":1162},"Quick start guides",{"href":1163,"dataGaName":1164,"dataGaLocation":1007},"/get-started/","quick setup checklists",{"text":761,"config":1166},{"href":763,"dataGaName":764,"dataGaLocation":1007},{"text":766,"config":1168},{"href":768,"dataGaName":769,"dataGaLocation":1007},{"text":771,"config":1170},{"href":773,"dataGaName":774,"dataGaLocation":1007},{"text":776,"config":1172},{"href":778,"dataGaName":779,"dataGaLocation":1007},{"text":781,"config":1174},{"href":783,"dataGaName":784,"dataGaLocation":1007},{"text":794,"config":1176},{"href":796,"dataGaName":797,"dataGaLocation":1007},{"text":799,"config":1178},{"href":801,"dataGaName":802,"dataGaLocation":1007},{"text":804,"config":1180},{"href":806,"dataGaName":807,"dataGaLocation":1007},{"text":809,"config":1182},{"href":811,"dataGaName":812,"dataGaLocation":1007},{"text":814,"config":1184},{"href":816,"dataGaName":817,"dataGaLocation":1007},{"text":877,"config":1186},{"href":879,"dataGaName":880,"dataGaLocation":1007},{"text":1188,"config":1189},"What's new",{"href":825,"dataGaName":826,"dataGaLocation":1007},{"text":1191,"config":1192},"All resources",{"href":755,"dataGaName":1193,"dataGaLocation":1007},"all resources",{"title":828,"links":1195},[1196,1198,1200,1202,1204,1206,1208,1210,1212,1216,1221],{"text":836,"config":1197},{"href":838,"dataGaName":830,"dataGaLocation":1007},{"text":841,"config":1199},{"href":843,"dataGaName":844,"dataGaLocation":1007},{"text":847,"config":1201},{"href":849,"dataGaName":850,"dataGaLocation":1007},{"text":852,"config":1203},{"href":854,"dataGaName":855,"dataGaLocation":1007},{"text":857,"config":1205},{"href":859,"dataGaName":860,"dataGaLocation":1007},{"text":862,"config":1207},{"href":864,"dataGaName":865,"dataGaLocation":1007},{"text":867,"config":1209},{"href":869,"dataGaName":870,"dataGaLocation":1007},{"text":872,"config":1211},{"href":874,"dataGaName":875,"dataGaLocation":1007},{"text":1213,"config":1214},"Sustainability",{"href":1215,"dataGaName":1213,"dataGaLocation":1007},"/sustainability/",{"text":1217,"config":1218},"Diversity, inclusion and belonging (DIB)",{"href":1219,"dataGaName":1220,"dataGaLocation":1007},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":1222,"config":1223},"Modern Slavery Transparency Statement",{"href":1224,"dataGaName":1225,"dataGaLocation":1007},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":1227},[1228,1231,1234],{"text":1229,"config":1230},"Terms",{"href":1037,"dataGaName":1038,"dataGaLocation":1007},{"text":1232,"config":1233},"Cookies",{"dataGaName":1047,"dataGaLocation":1007,"id":1048,"isOneTrustButton":289},{"text":1235,"config":1236},"Privacy",{"href":1042,"dataGaName":1043,"dataGaLocation":1007},[1238],{"id":1239,"title":7,"body":532,"config":1240,"content":1242,"description":532,"extension":1245,"meta":1246,"navigation":289,"path":1247,"seo":1248,"stem":1249,"__hash__":1250},"blogAuthors/en-us/blog/authors/sara-kassabian.yml",{"template":1241},"BlogAuthor",{"name":7,"config":1243},{"headshot":114,"ctfId":1244},"skassabian","yml",{},"/en-us/blog/authors/sara-kassabian",{},"en-us/blog/authors/sara-kassabian","6cCCPpjzkCDfb77tMVCyG8_FImRVQXA2VCMyiAjhFX0",[1252,1260,1268],{"title":1253,"description":1254,"heroImage":1255,"category":528,"date":1256,"authors":1257,"slug":1259,"externalUrl":532},"Two front doors: Module-level access in a Django GRC app","A technical deep-dive on restructuring a Django internal tool to support two distinct GitLab teams: Security Compliance and Internal Audit.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1750097663/Blog/Hero%20Images/Blog/Hero%20Images/Workflow%201800x945_2gQoQIbY9NvjLFpXtsxtXy_1750097663612.png","2026-10-05",[1258],"Donovan Felton","module-level-access-in-a-django-grc-app",{"title":1261,"description":1262,"heroImage":1263,"category":528,"date":1264,"authors":1265,"slug":1267,"externalUrl":532},"How GitLab reduced code-per-agentic-flow ratio by 45%","Learn how Flow Registry, a component-based framework that compiles YAML configurations into fully functional LangGraph flows, helped improve scalability and more.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1774465167/n5hlvrsrheadeccyr1oz.png","2026-09-22",[1266],"Mikolaj Wawrzyniak","how-gitlab-reduced-code-per-agentic-flow-ratio",{"title":1269,"description":1270,"heroImage":1271,"category":528,"date":1272,"authors":1273,"slug":1275,"externalUrl":532},"Co-Create: Building GitLab with our users","How GitLab users collaborated with us to build practical product improvements in the first half of 2026.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1750099203/Blog/Hero%20Images/Blog/Hero%20Images/blog-image-template-1800x945%20%2820%29_2bJGC5ZP3WheoqzlLT05C5_1750099203484.png","2026-09-10",[1274],"Isa Huerga","co-create-h1-2026",{"header":1277,"text":1278,"button":1282,"disclaimer":1285},"Start your free\n\n30-day GitLab trial\n",[1279,1280,1281],"GitLab Duo Agent Platform access","Try our most advanced features","Automate complex tasks with AI",{"text":555,"config":1283},{"href":1284,"dataGaName":558,"dataGaLocation":797},"https://gitlab.com/-/trials/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/blog/","No credit card required.",{"header":1287,"blurb":1288,"button":1289,"secondaryButton":1294},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":1290,"config":1291},"Get your free trial",{"href":1292,"dataGaName":558,"dataGaLocation":1293},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":1023,"config":1295},{"href":891,"dataGaName":563,"dataGaLocation":1293},1794911773373]