Skip to content
/ ci Public
forked from tlc-pack/ci

Repository which handles configuration of TVM CI infrastructure.

License

Apache-2.0 license
0 stars 11 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

naut-brian/ci

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

244 Commits
.github/workflows
.github/workflows
 
 
jenkins
jenkins
 
 
monitoring
monitoring
 
 
packer
packer
 
 
terraform
terraform
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
lint_local.py
lint_local.py
 
 

Repository files navigation

TVM Upstream CI

This repository holds the configuration as code for the CI of the upstream TVM project hosted on GitHub. Specifically, this repository currently handles configuration of a public Jenkins instance, as well as a dedicated set of worker nodes--this Jenkins instance is located at https://ci.tlcpack.ai

For issues and bug reports, please send them to https://github.com/apache/tvm/issues instead of this repo.

CI Diagram

This details the individual parts that interact in TVM's CI. For details on operations, see https://github.com/tlc-pack/ci.

graph TD
    Commit --> GitHub
    GitHub --> |`push` webhook| WebhookServer(Webhook Server)
    JobExecutor(Job Executor)
    WebhookServer --> JobExecutor
    JobExecutor -->  EC2Fleet(EC2 Fleet Plugin)
    EC2Fleet --> |capacity request| EC2(EC2 Autoscaler)
    JobExecutor --> WorkerEC2Instance
    Docker --> |build cache, artifacts| S3
    WorkerEC2Instance --> Docker
    Docker --> |docker pull| G(Docker Hub)
    Docker --> |docker push / pull| ECR
    Docker --> |Execute jobs| CIScripts(CI Scripts)
    RepoCITerraform(ci-terraform repo) --> |terraform| ECR
    RepoCITerraform(ci-terraform repo) --> |terraform| EC2
    RepoCITerraform(ci-terraform repo) --> |terraform| S3
    RepoCI(ci repo) --> |configuration via Ansible| WorkerEC2Instance
    RepoCIPacker(ci-packer) --> |AMIs| EC2
    Monitoring_Scrapers(Jenkins Scraper) --> Monitoring_DB(Postrgres)
    Grafana --> Monitoring_DB
    GitHub --> Windows
    GitHub --> MacOS

    Developers --> |check PR status|JenkinsUI(Jenkins Web UI)
    Monitoring_Scrapers --> |fetch job data| JenkinsUI
    Developers --> |git push| Commit
    Developers --> |create PR| GitHub

    subgraph Jenkins Head Node
        WebhookServer
        JobExecutor
        EC2Fleet
        JenkinsUI
    end

    subgraph GitHub Actions
        Windows
        MacOS
    end

    subgraph Configuration / Terraform
        RepoCITerraform
        RepoCI
        RepoCIPacker
    end

    subgraph Monitoring
        Monitoring_DB
        Grafana
        Monitoring_Scrapers
    end

    subgraph AWS
        subgraph Jenkins Workers
            WorkerEC2Instance(Worker EC2 Instance)
            subgraph "Worker EC2 Instance"
                Docker
                CIScripts
            end
        end
        EC2
        ECR
        S3
    end

Repository Layout

  • jenkins - configuration for the Jenkins head node
  • terraform - Terraform code to provision CI resources in AWS
  • packer - Packer configurations for AWS AMIs

Deploying Jenkins

Restarting Jenkins is an occasional but necessary service interruption. To minimize developer impact when updating TVM's Jenkins, follow these steps:

  1. Notify users:

    1. message the TVM Discord with a couple hours notice
    PSA that we'll be restarting Jenkins soon to <insert reason> -- we will need to retrigger in-flight builds as part of this process, so expect CI slowdowns for the next few hours.
    1. In Jenkins under Manage Jenkins > Configure System > System Message set it to something like