[{"data":1,"prerenderedAt":1278},["ShallowReactive",2],{"/blog/continuous-machine-learning-development-with-gitlab-ci":3,"navigation-en-us":528,"banner-en-us":964,"footer-en-us":972,"blog-post-authors-en-us-Dr. Elle O'Brien":1218,"blog-related-posts-en-us-continuous-machine-learning-development-with-gitlab-ci":1234,"blog-share-wrapper-en-us":1258,"next-steps-en-us":1268},{"id":4,"title":5,"authors":6,"body":8,"category":505,"date":506,"description":507,"extension":508,"externalUrl":509,"faq":509,"featured":510,"heroImage":511,"hidePromo":510,"meta":512,"navigation":129,"path":513,"seo":514,"slug":518,"stem":519,"tags":520,"template":526,"updatedDate":509,"__hash__":527},"blogPosts/en-us/blog/continuous-machine-learning-development-with-gitlab-ci.md","How machine learning ops works with GitLab and continuous machine learning",[7],"Dr. Elle O'Brien",{"type":9,"value":10,"toc":497},"minimark",[11,15,24,29,49,62,66,69,77,85,228,237,262,265,268,272,279,431,439,446,455,459,462,473,478,493],[12,13,14],"p",{},"Continuous integration (CI) is standard practice in software development for speeding up development cycles, and for keeping them short and painless. CI means making small commits, often, and automating tests so every commit is a release candidate.",[12,16,17,18,23],{},"When a project involves machine learning (ML), though, new challenges arise: Traditional ",[19,20,22],"a",{"href":21},"/topics/version-control/","version control systems"," (like Git) that are key to CI struggle to manage large datasets and models.\nFurthermore, typical pass-fail tests are too coarse for understanding ML model performance – you might need to consider how several metrics, like accuracy, sensitivity, and specificity, are affected by changes in your code or data.\nData visualizations like confusion matrices and loss plots are needed to make sense of the high-dimensional and often unintuitive behavior of models.",[25,26,28],"h2",{"id":27},"continuous-machine-learning-an-introduction","Continuous machine learning: an introduction",[12,30,31,36,37,42,43,48],{},[19,32,35],{"href":33,"rel":34},"https://iterative.ai",[],"Iterative.ai",", the team behind the popular open source version control system for ML projects ",[19,38,41],{"href":39,"rel":40},"https://dvc.org",[],"DVC"," (short for Data Version Control),\nhas recently released another open source project called ",[19,44,47],{"href":45,"rel":46},"https://cml.dev",[],"CML",", which stands for continuous machine learning.\nCML is our approach to adapting powerful CI systems like GitLab CI to common data science and ML use cases, including:",[50,51,52,56,59],"ul",{},[53,54,55],"li",{},"Automatic model training",[53,57,58],{},"Automatic model and dataset testing",[53,60,61],{},"Transparent and rich reporting about models and datasets (with data viz and metrics) in a merge request (MR)",[25,63,65],{"id":64},"your-first-continuous-machine-learning-report","Your first continuous machine learning report",[12,67,68],{},"CML helps you put tables, data viz, and even sample outputs from models into comments on your MRs, so you can review datasets and models like code.\nLet's see how to produce a basic report – we'll train an ML model using GitLab CI, and then report a model metric and confusion matrix in our MR.",[12,70,71,76],{},[72,73],"img",{"alt":74,"src":75},"Confusion Matrix","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398083/blog/Content%20Images/cml_confusion_matrix.jpg","\nConfusion matrix",[12,78,79,80,84],{},"To make this report, our ",[81,82,83],"code",{},".gitlab-ci.yml"," contains the following workflow:",[86,87,92],"pre",{"className":88,"code":89,"language":90,"meta":91,"style":91},"language-yaml shiki shiki-themes github-light","# .gitlab-ci.yml\nstages:\n    - cml_run\n\ncml:\n    stage: cml_run\n    image: dvcorg/cml-py3:latest\n\n    script:\n        - pip3 install -r requirements.txt\n        - python train.py\n\n        - cat metrics.txt >> report.md\n        - echo >> report.md\n        - cml-publish confusion_matrix.png --md --title 'confusion-matrix' >> report.md\n        - cml-send-comment report.md\n\n","yaml","",[81,93,94,103,114,124,131,139,150,161,166,174,183,191,196,204,212,220],{"__ignoreMap":91},[95,96,99],"span",{"class":97,"line":98},"line",1,[95,100,102],{"class":101},"sAwPA","# .gitlab-ci.yml\n",[95,104,106,110],{"class":97,"line":105},2,[95,107,109],{"class":108},"shJU0","stages",[95,111,113],{"class":112},"sgsFI",":\n",[95,115,117,120],{"class":97,"line":116},3,[95,118,119],{"class":112},"    - ",[95,121,123],{"class":122},"sYBdl","cml_run\n",[95,125,127],{"class":97,"line":126},4,[95,128,130],{"emptyLinePlaceholder":129},true,"\n",[95,132,134,137],{"class":97,"line":133},5,[95,135,136],{"class":108},"cml",[95,138,113],{"class":112},[95,140,142,145,148],{"class":97,"line":141},6,[95,143,144],{"class":108},"    stage",[95,146,147],{"class":112},": ",[95,149,123],{"class":122},[95,151,153,156,158],{"class":97,"line":152},7,[95,154,155],{"class":108},"    image",[95,157,147],{"class":112},[95,159,160],{"class":122},"dvcorg/cml-py3:latest\n",[95,162,164],{"class":97,"line":163},8,[95,165,130],{"emptyLinePlaceholder":129},[95,167,169,172],{"class":97,"line":168},9,[95,170,171],{"class":108},"    script",[95,173,113],{"class":112},[95,175,177,180],{"class":97,"line":176},10,[95,178,179],{"class":112},"        - ",[95,181,182],{"class":122},"pip3 install -r requirements.txt\n",[95,184,186,188],{"class":97,"line":185},11,[95,187,179],{"class":112},[95,189,190],{"class":122},"python train.py\n",[95,192,194],{"class":97,"line":193},12,[95,195,130],{"emptyLinePlaceholder":129},[95,197,199,201],{"class":97,"line":198},13,[95,200,179],{"class":112},[95,202,203],{"class":122},"cat metrics.txt >> report.md\n",[95,205,207,209],{"class":97,"line":206},14,[95,208,179],{"class":112},[95,210,211],{"class":122},"echo >> report.md\n",[95,213,215,217],{"class":97,"line":214},15,[95,216,179],{"class":112},[95,218,219],{"class":122},"cml-publish confusion_matrix.png --md --title 'confusion-matrix' >> report.md\n",[95,221,223,225],{"class":97,"line":222},16,[95,224,179],{"class":112},[95,226,227],{"class":122},"cml-send-comment report.md\n",[12,229,230,231,236],{},"The entire ",[19,232,235],{"href":233,"rel":234},"https://gitlab.com/iterative.ai/cml-base-case/",[],"project repository is available here",".\nThe steps consist of the following:",[50,238,239,250,256],{},[53,240,241,245,246,249],{},[242,243,244],"strong",{},"Train",": This is a classic training step where we install requirements (like ",[81,247,248],{},"pip"," packages) and run the training script.",[53,251,252,255],{},[242,253,254],{},"Write a CML report",": Produced metrics are appended to a markdown report.",[53,257,258,261],{},[242,259,260],{},"Publish a CML report",": CML publishes an image of the confusion matrix with the embedded metrics to your GitLab MR.",[12,263,264],{},"Now, when you and your teammates are deciding if your changes have had a positive effect on your modeling goals,\nyou have a dashboard of sorts to review. Plus, this report is linked by Git to your exact project version (data and code) and the runner used for training and the logs from that run.",[12,266,267],{},"This is the simplest use case for achieving continuous machine learning with CML and GitLab. In the next section we'll look at a more complex use case.",[25,269,271],{"id":270},"cml-with-dvc-for-data-version-control","CML with DVC for data version control",[12,273,274,275,278],{},"In machine learning projects, you need to track changes in your datasets as well as changes in your code.\nSince Git is frequently a poor fit for managing large files, we can use ",[19,276,41],{"href":39,"rel":277},[]," to link remote datasets to your CI system.",[86,280,282],{"className":88,"code":281,"language":90,"meta":91,"style":91},"# .gitlab-ci.yml\nstages:\n  - cml_run\n\ncml:\n  stage: cml_run\n  image: dvcorg/cml-py3:latest\n  script:\n    - dvc pull data\n\n    - pip install -r requirements.txt\n    - dvc repro\n\n    # Compare metrics to master\n    - git fetch --prune\n    - dvc metrics diff --show-md master >> report.md\n    - echo >> report.md\n\n    # Visualize loss function diff\n    - dvc plots diff\n      --target loss.csv --show-vega master > vega.json\n    - vl2png vega.json | cml-publish --md >> report.md\n    - cml-send-comment report.md\n\n",[81,283,284,288,294,301,305,311,320,329,336,343,347,354,361,365,370,377,384,391,396,402,410,416,424],{"__ignoreMap":91},[95,285,286],{"class":97,"line":98},[95,287,102],{"class":101},[95,289,290,292],{"class":97,"line":105},[95,291,109],{"class":108},[95,293,113],{"class":112},[95,295,296,299],{"class":97,"line":116},[95,297,298],{"class":112},"  - ",[95,300,123],{"class":122},[95,302,303],{"class":97,"line":126},[95,304,130],{"emptyLinePlaceholder":129},[95,306,307,309],{"class":97,"line":133},[95,308,136],{"class":108},[95,310,113],{"class":112},[95,312,313,316,318],{"class":97,"line":141},[95,314,315],{"class":108},"  stage",[95,317,147],{"class":112},[95,319,123],{"class":122},[95,321,322,325,327],{"class":97,"line":152},[95,323,324],{"class":108},"  image",[95,326,147],{"class":112},[95,328,160],{"class":122},[95,330,331,334],{"class":97,"line":163},[95,332,333],{"class":108},"  script",[95,335,113],{"class":112},[95,337,338,340],{"class":97,"line":168},[95,339,119],{"class":112},[95,341,342],{"class":122},"dvc pull data\n",[95,344,345],{"class":97,"line":176},[95,346,130],{"emptyLinePlaceholder":129},[95,348,349,351],{"class":97,"line":185},[95,350,119],{"class":112},[95,352,353],{"class":122},"pip install -r requirements.txt\n",[95,355,356,358],{"class":97,"line":193},[95,357,119],{"class":112},[95,359,360],{"class":122},"dvc repro\n",[95,362,363],{"class":97,"line":198},[95,364,130],{"emptyLinePlaceholder":129},[95,366,367],{"class":97,"line":206},[95,368,369],{"class":101},"    # Compare metrics to master\n",[95,371,372,374],{"class":97,"line":214},[95,373,119],{"class":112},[95,375,376],{"class":122},"git fetch --prune\n",[95,378,379,381],{"class":97,"line":222},[95,380,119],{"class":112},[95,382,383],{"class":122},"dvc metrics diff --show-md master >> report.md\n",[95,385,387,389],{"class":97,"line":386},17,[95,388,119],{"class":112},[95,390,211],{"class":122},[95,392,394],{"class":97,"line":393},18,[95,395,130],{"emptyLinePlaceholder":129},[95,397,399],{"class":97,"line":398},19,[95,400,401],{"class":101},"    # Visualize loss function diff\n",[95,403,405,407],{"class":97,"line":404},20,[95,406,119],{"class":112},[95,408,409],{"class":122},"dvc plots diff\n",[95,411,413],{"class":97,"line":412},21,[95,414,415],{"class":122},"      --target loss.csv --show-vega master > vega.json\n",[95,417,419,421],{"class":97,"line":418},22,[95,420,119],{"class":112},[95,422,423],{"class":122},"vl2png vega.json | cml-publish --md >> report.md\n",[95,425,427,429],{"class":97,"line":426},23,[95,428,119],{"class":112},[95,430,227],{"class":122},[12,432,230,433,438],{},[19,434,437],{"href":435,"rel":436},"https://gitlab.com/iterative.ai/cml-dvc-case",[],"project is available here",".\nIn this workflow, we have additional steps that use DVC to pull a training dataset, run an experiment, and then use CML to publish the report in your MR.",[12,440,441,445],{},[72,442],{"alt":443,"src":444},"CML with DVC","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398083/blog/Content%20Images/cml_dvc.jpg","\nCML with DVC",[12,447,448,449,454],{},"For more details about ML data versioning and tracking, check out the ",[19,450,453],{"href":451,"rel":452},"https://dvc.org/doc",[],"DVC documentation",".",[25,456,458],{"id":457},"summary","Summary",[12,460,461],{},"We made CML to adapt CI to machine learning, so data science teams can enjoy benefits such as:",[50,463,464,467,470],{},[53,465,466],{},"Your code, data, models, and training infrastructure (hardware and software environment) will be Git versioned.",[53,468,469],{},"You’re automating work, testing frequently, and getting fast feedback (with visual reports if you use CML). In the long run, this will almost certainly speed up your project’s development.",[53,471,472],{},"CI systems make your work visible to everyone on your team. No one has to search very hard to find the code, data, and model from your best run.",[474,475,477],"h3",{"id":476},"about-the-guest-author","About the guest author",[12,479,480],{},[481,482,483,484,487,488,454],"em",{},"Dr. Elle O'Brien is a Ph.D data scientist at iterative.ai and co-creator of ",[19,485,47],{"href":45,"rel":486},[]," project. She is also a lecturer at ",[19,489,492],{"href":490,"rel":491},"https://www.si.umich.edu/",[],"UMSI",[494,495,496],"style",{},"html pre.shiki code .sAwPA, html code.shiki .sAwPA{--shiki-default:#6A737D}html pre.shiki code .shJU0, html code.shiki .shJU0{--shiki-default:#22863A}html pre.shiki code .sgsFI, html code.shiki .sgsFI{--shiki-default:#24292E}html pre.shiki code .sYBdl, html code.shiki .sYBdl{--shiki-default:#032F62}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":91,"searchDepth":105,"depth":105,"links":498},[499,500,501,502],{"id":27,"depth":105,"text":28},{"id":64,"depth":105,"text":65},{"id":270,"depth":105,"text":271},{"id":457,"depth":105,"text":458,"children":503},[504],{"id":476,"depth":116,"text":477},"news","2020-12-01","We share different machine learning use cases for CML projects using GitLab CI.","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749681802/Blog/Hero%20Images/gitlab_cml_dvc_banner.png",{},"/en-us/blog/continuous-machine-learning-development-with-gitlab-ci",{"title":5,"description":507,"ogTitle":5,"ogDescription":507,"noIndex":510,"ogImage":511,"ogUrl":515,"ogSiteName":516,"ogType":517,"canonicalUrls":515},"https://about.gitlab.com/blog/continuous-machine-learning-development-with-gitlab-ci","https://about.gitlab.com","article","continuous-machine-learning-development-with-gitlab-ci","en-us/blog/continuous-machine-learning-development-with-gitlab-ci",[521,522,523,524,525],"CI/CD","community","integrations","open source","AI","BlogPost","muzkwTo8KWJrVRTjRsBHZhJuABpj2SMOj8I8yiQeU5Y",{"logo":529,"freeTrial":534,"sales":539,"login":544,"items":549,"search":883,"minimal":915,"duo":934,"switchNav":943,"pricingDeployment":954},{"config":530},{"href":531,"dataGaName":532,"dataGaLocation":533},"/","gitlab logo","header",{"text":535,"config":536},"Get free trial",{"href":537,"dataGaName":538,"dataGaLocation":533},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":540,"config":541},"Request a demo",{"href":542,"dataGaName":543,"dataGaLocation":533},"/request-a-demo/","sales",{"text":545,"config":546},"Sign in",{"href":547,"dataGaName":548,"dataGaLocation":533},"https://gitlab.com/users/sign_in/","sign in",[550,635,723,728,807,861],{"text":551,"left":129,"config":552,"menu":554},"Platform",{"dataNavLevelOne":553},"platform",{"type":555,"link":556,"columns":560,"feature":624},"lists",{"text":557,"config":558},"Explore the Platform",{"href":559,"dataGaName":553,"dataGaLocation":533},"/platform/",[561,583,606],{"title":562,"items":563},"DevOps",[564,568,573,578],{"text":521,"config":565},{"href":566,"dataGaLocation":533,"dataGaName":567},"/solutions/continuous-integration/","ci/cd",{"text":569,"config":570},"Source Code Management",{"href":571,"dataGaLocation":533,"dataGaName":572},"/solutions/source-code-management/","source code management",{"text":574,"config":575},"Agile Delivery",{"href":576,"dataGaLocation":533,"dataGaName":577},"/solutions/agile-delivery/","agile delivery",{"text":579,"config":580},"Artifact Management",{"href":581,"dataGaLocation":533,"dataGaName":582},"/artifact-central/","artifact management",{"title":584,"items":585},"Security & Governance",[586,591,596,601],{"text":587,"config":588},"Application Security Testing",{"href":589,"dataGaLocation":533,"dataGaName":590},"/solutions/application-security-testing/","application security testing",{"text":592,"config":593},"Governance & Compliance",{"href":594,"dataGaLocation":533,"dataGaName":595},"/solutions/software-compliance/","governance and compliance",{"text":597,"config":598},"Secrets Management",{"href":599,"dataGaLocation":533,"dataGaName":600},"/gitlab-secrets-manager/","secrets management",{"text":602,"config":603},"Supply Chain Security",{"href":604,"dataGaLocation":533,"dataGaName":605},"/solutions/supply-chain/","supply chain security",{"title":607,"items":608},"Agentic AI & Context",[609,614,619],{"text":610,"config":611},"Duo Agent Platform",{"href":612,"dataGaLocation":533,"dataGaName":613},"/gitlab-duo-agent-platform/","duo agent platform",{"text":615,"config":616},"Orbit Context Graph",{"href":617,"dataGaLocation":533,"dataGaName":618},"/gitlab-orbit/","orbit",{"text":620,"config":621},"Visibility & Measurement",{"href":622,"dataGaLocation":533,"dataGaName":623},"/solutions/visibility-measurement/","visibility and measurement",{"config":625,"title":628,"text":629,"link":630},{"background":626,"textColor":627},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1790185102/o7dn32anqcb6z8dgd3as.jpg')","#000","GitLab Orbit is generally available","Higher agent accuracy, lower token cost",{"text":631,"config":632},"Read the announcement",{"href":633,"dataGaName":634,"dataGaLocation":533},"/blog/gitlab-orbit-ga/","orbit ga announcement",{"text":636,"left":129,"config":637,"menu":639},"Solutions",{"dataNavLevelOne":638},"solutions",{"type":555,"link":640,"columns":644,"feature":714},{"text":641,"config":642},"View all Solutions",{"href":643,"dataGaName":638,"dataGaLocation":533},"/solutions/",[645,663,681],{"title":646,"items":647},"Outcomes",[648,653,658],{"text":649,"config":650},"DevOps Modernization",{"href":651,"dataGaLocation":533,"dataGaName":652},"/assessments/devops-modernization-assessment/","devops modernization",{"text":654,"config":655},"Security Modernization",{"href":656,"dataGaLocation":533,"dataGaName":657},"/assessments/security-modernization-assessment/","security modernization",{"text":659,"config":660},"AI Modernization",{"href":661,"dataGaLocation":533,"dataGaName":662},"/assessments/ai-modernization-assessment/","ai modernization",{"title":664,"items":665},"By size",[666,671,676],{"text":667,"config":668},"Enterprise",{"href":669,"dataGaLocation":533,"dataGaName":670},"/enterprise/","enterprise",{"text":672,"config":673},"Small Business",{"href":674,"dataGaLocation":533,"dataGaName":675},"/small-business/","small business",{"text":677,"config":678},"Startups",{"href":679,"dataGaLocation":533,"dataGaName":680},"/solutions/startups/","startups",{"title":682,"items":683},"Industries",[684,689,694,699,704,709],{"text":685,"config":686},"Financial Services",{"href":687,"dataGaLocation":533,"dataGaName":688},"/solutions/finance/","financial services",{"text":690,"config":691},"Public Sector",{"href":692,"dataGaLocation":533,"dataGaName":693},"/solutions/public-sector/","public sector",{"text":695,"config":696},"Telecommunications",{"href":697,"dataGaLocation":533,"dataGaName":698},"/solutions/telecommunications/","telecommunications",{"text":700,"config":701},"Automotive",{"href":702,"dataGaLocation":533,"dataGaName":703},"/solutions/automotive/","automotive",{"text":705,"config":706},"Education",{"href":707,"dataGaLocation":533,"dataGaName":708},"/solutions/education/","education",{"text":710,"config":711},"Aerospace",{"href":712,"dataGaLocation":533,"dataGaName":713},"/solutions/aerospace/","aerospace",{"config":715,"title":716,"text":717,"link":718},{"background":626,"textColor":627},"GitLab Security Standard","Learn how to harden your software factory and let AI agents earn autonomy, without trading away security.",{"text":719,"config":720},"Explore the standard",{"href":721,"dataGaName":722,"dataGaLocation":533},"/gitlab-security-standard/","gitlab security standard",{"text":724,"config":725},"Pricing",{"href":726,"dataGaName":727,"dataGaLocation":533,"dataNavLevelOne":727},"/pricing/","pricing",{"text":729,"config":730,"menu":732},"Resources",{"dataNavLevelOne":731},"resources",{"type":555,"link":733,"columns":737,"feature":798},{"text":734,"config":735},"View all resources",{"href":736,"dataGaName":731,"dataGaLocation":533},"/resources/",[738,771],{"title":739,"items":740},"Discover",[741,746,751,756,761,766],{"text":742,"config":743},"Docs",{"href":744,"dataGaName":745,"dataGaLocation":533},"https://docs.gitlab.com/","docs",{"text":747,"config":748},"University",{"href":749,"dataGaName":750,"dataGaLocation":533},"https://university.gitlab.com/","university",{"text":752,"config":753},"Demo Series",{"href":754,"dataGaName":755,"dataGaLocation":533},"/gitlab-technical-demo-series/","demo series",{"text":757,"config":758},"Demo Hub",{"href":759,"dataGaName":760,"dataGaLocation":533},"/demo-hub/","demo hub",{"text":762,"config":763},"Services",{"href":764,"dataGaName":765,"dataGaLocation":533},"/services/","services",{"text":767,"config":768},"Well Architected",{"href":769,"dataGaName":770,"dataGaLocation":533},"/resources/well-architected/","well architected",{"title":772,"items":773},"Connect",[774,779,783,788,793],{"text":775,"config":776},"Blog",{"href":777,"dataGaName":778,"dataGaLocation":533},"/blog/","blog",{"text":780,"config":781},"Community",{"href":782,"dataGaName":522,"dataGaLocation":533},"/community/",{"text":784,"config":785},"Customers",{"href":786,"dataGaName":787,"dataGaLocation":533},"/customers/","customers",{"text":789,"config":790},"Partners",{"href":791,"dataGaName":792,"dataGaLocation":533},"/partners/","partners",{"text":794,"config":795},"Events",{"href":796,"dataGaName":797,"dataGaLocation":533},"/events/","events",{"config":799,"title":800,"text":801,"link":802},{"background":626,"textColor":627},"What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":803,"config":804},"Read the latest",{"href":805,"dataGaName":806,"dataGaLocation":533},"/whats-new/","whats new",{"text":808,"config":809,"menu":811},"Company",{"dataNavLevelOne":810},"company",{"type":555,"columns":812},[813],{"items":814},[815,820,826,831,836,841,846,851,856],{"text":816,"config":817},"About",{"href":818,"dataGaName":819,"dataGaLocation":533},"/company/","about",{"text":821,"config":822,"footerGa":825},"Jobs",{"href":823,"dataGaName":824,"dataGaLocation":533},"/jobs/","jobs",{"dataGaName":824},{"text":827,"config":828},"Press",{"href":829,"dataGaName":830,"dataGaLocation":533},"/press/","press",{"text":832,"config":833},"Handbook",{"href":834,"dataGaName":835,"dataGaLocation":533},"https://handbook.gitlab.com/","handbook",{"text":837,"config":838},"Leadership",{"href":839,"dataGaName":840,"dataGaLocation":533},"/company/team/e-group/","leadership",{"text":842,"config":843},"Investor relations",{"href":844,"dataGaName":845,"dataGaLocation":533},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":847,"config":848},"Trust Center",{"href":849,"dataGaName":850,"dataGaLocation":533},"/security/","trust center",{"text":852,"config":853},"AI Transparency Center",{"href":854,"dataGaName":855,"dataGaLocation":533},"/ai-transparency-center/","ai transparency center",{"text":857,"config":858},"Newsletter",{"href":859,"dataGaName":860,"dataGaLocation":533},"/company/contact/#contact-forms","newsletter",{"text":862,"config":863,"menu":864},"Contact us",{"dataNavLevelOne":810},{"type":555,"columns":865},[866],{"items":867},[868,873,878],{"text":869,"config":870},"Talk to sales",{"href":871,"dataGaName":872,"dataGaLocation":533},"/sales/","talk to sales",{"text":874,"config":875},"Support portal",{"href":876,"dataGaName":877,"dataGaLocation":533},"https://support.gitlab.com/hc/en-us","support portal",{"text":879,"config":880},"Customer portal",{"href":881,"dataGaName":882,"dataGaLocation":533},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":884,"login":885,"suggestions":892},"Close",{"text":886,"link":887},"To search repositories and projects, login to",{"text":888,"config":889},"gitlab.com",{"href":547,"dataGaName":890,"dataGaLocation":891},"search login","search",{"text":893,"default":894},"Suggestions",[895,898,901,903,907,911],{"text":896,"config":897},"GitLab Duo Agent Platform",{"href":612,"dataGaName":896,"dataGaLocation":891},{"text":899,"config":900},"Code Suggestions (AI)",{"href":612,"dataGaName":899,"dataGaLocation":891},{"text":521,"config":902},{"href":566,"dataGaName":521,"dataGaLocation":891},{"text":904,"config":905},"GitLab on AWS",{"href":906,"dataGaName":904,"dataGaLocation":891},"/partners/technology-partners/aws/",{"text":908,"config":909},"GitLab on Google Cloud",{"href":910,"dataGaName":908,"dataGaLocation":891},"/partners/technology-partners/google-cloud-platform/",{"text":912,"config":913},"Why GitLab?",{"href":914,"dataGaName":912,"dataGaLocation":891},"/why-gitlab/",{"freeTrial":916,"mobileIcon":921,"desktopIcon":926,"secondaryButton":929},{"text":917,"config":918},"Start free trial",{"href":919,"dataGaName":538,"dataGaLocation":920},"https://gitlab.com/-/trials/new/","nav",{"altText":922,"config":923},"Gitlab Icon",{"src":924,"dataGaName":925,"dataGaLocation":920},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":922,"config":927},{"src":928,"dataGaName":925,"dataGaLocation":920},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":930,"config":931},"Get Started",{"href":932,"dataGaName":933,"dataGaLocation":920},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":935,"mobileIcon":939,"desktopIcon":941},{"text":936,"config":937},"Learn more about GitLab Duo",{"href":612,"dataGaName":938,"dataGaLocation":920},"gitlab duo",{"altText":922,"config":940},{"src":924,"dataGaName":925,"dataGaLocation":920},{"altText":922,"config":942},{"src":928,"dataGaName":925,"dataGaLocation":920},{"button":944,"mobileIcon":949,"desktopIcon":951},{"text":945,"config":946},"/switch",{"href":947,"dataGaName":948,"dataGaLocation":920},"#contact","switch",{"altText":922,"config":950},{"src":924,"dataGaName":925,"dataGaLocation":920},{"altText":922,"config":952},{"src":953,"dataGaName":925,"dataGaLocation":920},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":955,"mobileIcon":960,"desktopIcon":962},{"text":956,"config":957},"Back to pricing",{"href":726,"dataGaName":958,"dataGaLocation":920,"icon":959},"back to pricing","GoBack",{"altText":922,"config":961},{"src":924,"dataGaName":925,"dataGaLocation":920},{"altText":922,"config":963},{"src":928,"dataGaName":925,"dataGaLocation":920},{"title":965,"button":966,"config":970},"GitLab Orbit is generally available. Increase agent accuracy and lower token cost",{"text":631,"config":967},{"href":633,"dataGaName":968,"dataGaLocation":969},"orbit-ga-banner","global-banner",{"layout":971,"disabled":510},"release",{"data":973},{"text":974,"config":975,"items":980,"minimal":1207},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"twitter":976,"facebook":977,"youtube":978,"linkedin":979},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[981,1029,1087,1134,1175],{"title":724,"links":982,"subMenu":998},[983,988,993],{"text":984,"config":985},"View plans",{"href":726,"dataGaName":986,"dataGaLocation":987},"view plans","footer",{"text":989,"config":990},"Why Premium?",{"href":991,"dataGaName":992,"dataGaLocation":987},"/pricing/premium/","why premium",{"text":994,"config":995},"Why Ultimate?",{"href":996,"dataGaName":997,"dataGaLocation":987},"/pricing/ultimate/","why ultimate",[999],{"title":1000,"links":1001},"Contact Us",[1002,1005,1007,1009,1014,1019,1024],{"text":1003,"config":1004},"Contact sales",{"href":871,"dataGaName":543,"dataGaLocation":987},{"text":874,"config":1006},{"href":876,"dataGaName":877,"dataGaLocation":987},{"text":879,"config":1008},{"href":881,"dataGaName":882,"dataGaLocation":987},{"text":1010,"config":1011},"Status",{"href":1012,"dataGaName":1013,"dataGaLocation":987},"https://status.gitlab.com/","status",{"text":1015,"config":1016},"Terms of use",{"href":1017,"dataGaName":1018,"dataGaLocation":987},"/terms/","terms of use",{"text":1020,"config":1021},"Privacy statement",{"href":1022,"dataGaName":1023,"dataGaLocation":987},"/privacy/","privacy statement",{"text":1025,"config":1026},"Cookie preferences",{"dataGaName":1027,"dataGaLocation":987,"id":1028,"isOneTrustButton":129},"cookie preferences","ot-sdk-btn",{"title":551,"links":1030,"subMenu":1046},[1031,1034,1036,1038,1040,1042],{"text":1032,"config":1033},"Explore the platform",{"href":559,"dataGaName":553,"dataGaLocation":987},{"text":610,"config":1035},{"href":612,"dataGaName":613,"dataGaLocation":987},{"text":521,"config":1037},{"href":566,"dataGaName":567,"dataGaLocation":987},{"text":615,"config":1039},{"href":617,"dataGaName":618,"dataGaLocation":987},{"text":597,"config":1041},{"href":599,"dataGaName":600,"dataGaLocation":987},{"text":1043,"config":1044},"Why GitLab",{"href":914,"dataGaName":1045,"dataGaLocation":987},"why gitlab",[1047],{"title":1048,"links":1049},"Topics",[1050,1054,1059,1063,1067,1072,1077,1082],{"text":521,"config":1051},{"href":1052,"dataGaName":1053,"dataGaLocation":987},"/topics/ci-cd/","cicd",{"text":1055,"config":1056},"GitOps",{"href":1057,"dataGaName":1058,"dataGaLocation":987},"/topics/gitops/","gitops",{"text":562,"config":1060},{"href":1061,"dataGaName":1062,"dataGaLocation":987},"/topics/devops/","devops",{"text":1064,"config":1065},"Version Control",{"href":21,"dataGaName":1066,"dataGaLocation":987},"version control",{"text":1068,"config":1069},"DevSecOps",{"href":1070,"dataGaName":1071,"dataGaLocation":987},"/topics/devsecops/","devsecops",{"text":1073,"config":1074},"Cloud Native",{"href":1075,"dataGaName":1076,"dataGaLocation":987},"/topics/cloud-native/","cloud native",{"text":1078,"config":1079},"AI for Coding",{"href":1080,"dataGaName":1081,"dataGaLocation":987},"/topics/devops/ai-for-coding/","ai for coding",{"text":1083,"config":1084},"Agentic AI",{"href":1085,"dataGaName":1086,"dataGaLocation":987},"/topics/agentic-ai/","agentic ai",{"title":636,"links":1088},[1089,1092,1095,1098,1101,1104,1106,1108,1110,1112,1114,1117,1119,1122,1126,1130],{"text":1090,"config":1091},"DevOps modernization",{"href":651,"dataGaName":652,"dataGaLocation":987},{"text":1093,"config":1094},"Security modernization",{"href":656,"dataGaName":657,"dataGaLocation":987},{"text":1096,"config":1097},"AI modernization",{"href":661,"dataGaName":662,"dataGaLocation":987},{"text":1099,"config":1100},"Financial services",{"href":687,"dataGaName":688,"dataGaLocation":987},{"text":1102,"config":1103},"Public sector",{"href":692,"dataGaName":693,"dataGaLocation":987},{"text":695,"config":1105},{"href":697,"dataGaName":698,"dataGaLocation":987},{"text":700,"config":1107},{"href":702,"dataGaName":703,"dataGaLocation":987},{"text":705,"config":1109},{"href":707,"dataGaName":708,"dataGaLocation":987},{"text":710,"config":1111},{"href":712,"dataGaName":713,"dataGaLocation":987},{"text":667,"config":1113},{"href":669,"dataGaName":670,"dataGaLocation":987},{"text":1115,"config":1116},"Small business",{"href":674,"dataGaName":675,"dataGaLocation":987},{"text":677,"config":1118},{"href":679,"dataGaName":680,"dataGaLocation":987},{"text":1055,"config":1120},{"href":1121,"dataGaName":1058,"dataGaLocation":987},"/solutions/gitops/",{"text":1123,"config":1124},"Software composition analysis",{"href":604,"dataGaName":1125,"dataGaLocation":987},"software composition analysis",{"text":1127,"config":1128},"Value stream management",{"href":622,"dataGaName":1129,"dataGaLocation":987},"value stream management",{"text":1131,"config":1132},"All solutions",{"href":643,"dataGaName":1133,"dataGaLocation":987},"all solutions",{"title":729,"links":1135},[1136,1141,1146,1148,1150,1152,1154,1156,1158,1160,1162,1164,1166,1168,1171],{"text":1137,"config":1138},"Install",{"href":1139,"dataGaName":1140,"dataGaLocation":987},"/install/","install",{"text":1142,"config":1143},"Quick start guides",{"href":1144,"dataGaName":1145,"dataGaLocation":987},"/get-started/","quick setup checklists",{"text":742,"config":1147},{"href":744,"dataGaName":745,"dataGaLocation":987},{"text":747,"config":1149},{"href":749,"dataGaName":750,"dataGaLocation":987},{"text":752,"config":1151},{"href":754,"dataGaName":755,"dataGaLocation":987},{"text":757,"config":1153},{"href":759,"dataGaName":760,"dataGaLocation":987},{"text":762,"config":1155},{"href":764,"dataGaName":765,"dataGaLocation":987},{"text":775,"config":1157},{"href":777,"dataGaName":778,"dataGaLocation":987},{"text":780,"config":1159},{"href":782,"dataGaName":522,"dataGaLocation":987},{"text":784,"config":1161},{"href":786,"dataGaName":787,"dataGaLocation":987},{"text":789,"config":1163},{"href":791,"dataGaName":792,"dataGaLocation":987},{"text":794,"config":1165},{"href":796,"dataGaName":797,"dataGaLocation":987},{"text":857,"config":1167},{"href":859,"dataGaName":860,"dataGaLocation":987},{"text":1169,"config":1170},"What's new",{"href":805,"dataGaName":806,"dataGaLocation":987},{"text":1172,"config":1173},"All resources",{"href":736,"dataGaName":1174,"dataGaLocation":987},"all resources",{"title":808,"links":1176},[1177,1179,1181,1183,1185,1187,1189,1191,1193,1197,1202],{"text":816,"config":1178},{"href":818,"dataGaName":810,"dataGaLocation":987},{"text":821,"config":1180},{"href":823,"dataGaName":824,"dataGaLocation":987},{"text":827,"config":1182},{"href":829,"dataGaName":830,"dataGaLocation":987},{"text":832,"config":1184},{"href":834,"dataGaName":835,"dataGaLocation":987},{"text":837,"config":1186},{"href":839,"dataGaName":840,"dataGaLocation":987},{"text":842,"config":1188},{"href":844,"dataGaName":845,"dataGaLocation":987},{"text":847,"config":1190},{"href":849,"dataGaName":850,"dataGaLocation":987},{"text":852,"config":1192},{"href":854,"dataGaName":855,"dataGaLocation":987},{"text":1194,"config":1195},"Sustainability",{"href":1196,"dataGaName":1194,"dataGaLocation":987},"/sustainability/",{"text":1198,"config":1199},"Diversity, inclusion and belonging (DIB)",{"href":1200,"dataGaName":1201,"dataGaLocation":987},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":1203,"config":1204},"Modern Slavery Transparency Statement",{"href":1205,"dataGaName":1206,"dataGaLocation":987},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":1208},[1209,1212,1215],{"text":1210,"config":1211},"Terms",{"href":1017,"dataGaName":1018,"dataGaLocation":987},{"text":1213,"config":1214},"Cookies",{"dataGaName":1027,"dataGaLocation":987,"id":1028,"isOneTrustButton":129},{"text":1216,"config":1217},"Privacy",{"href":1022,"dataGaName":1023,"dataGaLocation":987},[1219],{"id":1220,"title":1221,"body":509,"config":1222,"content":1224,"description":509,"extension":1228,"meta":1229,"navigation":129,"path":1230,"seo":1231,"stem":1232,"__hash__":1233},"blogAuthors/en-us/blog/authors/dr-elle-obrien.yml","Dr Elle Obrien",{"template":1223},"BlogAuthor",{"name":7,"config":1225},{"headshot":1226,"ctfId":1227},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749659488/Blog/Author%20Headshots/gitlab-logo-extra-whitespace.png","Dr-Elle-OBrien","yml",{},"/en-us/blog/authors/dr-elle-obrien",{},"en-us/blog/authors/dr-elle-obrien","2QbDS094UsB0v6BSWztBkgBs8iuHIAkelzXjLqPogvQ",[1235,1243,1251],{"title":1236,"description":1237,"heroImage":1238,"category":505,"date":1239,"authors":1240,"slug":1242,"externalUrl":509},"10 questions to ask any software vendor about sovereignty","If sovereignty is on your priority list, use these ten questions to find out whether your vendors take it as seriously as you do.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1785336708/ww4mlighwlvvzgnmbe6w.png","2026-11-02",[1241],"Bryan Ross","10-questions-to-ask-any-software-vendor-about-sovereignty",{"title":1244,"description":1245,"heroImage":1246,"category":505,"date":1247,"authors":1248,"slug":1250,"externalUrl":509},"Redesign how your team ships software with AI agents","A practical guide to AI-native engineering from practitioners: Find where delivery slows, decide where humans stay in control, and verify what ships.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1792421291/mfgvawlpuk9a2tdimsgn.png","2026-10-20",[1249],"Chandler Gibbons","redesign-how-your-team-ships-software-with-ai-agents",{"title":1252,"description":1253,"heroImage":1238,"category":505,"date":1254,"authors":1255,"slug":1257,"externalUrl":509},"When code is abundant","Producing code is getting cheap. Trusting it is not. Why enterprises need a durable layer of context, verification, and governance around AI agents.","2026-08-24",[1256],"Bill Staples","when-code-is-abundant",{"header":1259,"text":1260,"button":1264,"disclaimer":1267},"Start your free\n\n30-day GitLab trial\n",[1261,1262,1263],"GitLab Duo Agent Platform access","Try our most advanced features","Automate complex tasks with AI",{"text":535,"config":1265},{"href":1266,"dataGaName":538,"dataGaLocation":778},"https://gitlab.com/-/trials/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/blog/","No credit card required.",{"header":1269,"blurb":1270,"button":1271,"secondaryButton":1276},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":1272,"config":1273},"Get your free trial",{"href":1274,"dataGaName":538,"dataGaLocation":1275},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":1003,"config":1277},{"href":871,"dataGaName":543,"dataGaLocation":1275},1794911782755]