|
|
|
|
@@ -69,30 +69,41 @@ To migrate your repositories from {% data variables.product.prodname_ghe_server
|
|
|
|
|
|
|
|
|
|
## Step 3: Set up blob storage
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.blob-storage-intro %}
|
|
|
|
|
When performing a repository migration, you must store your repository data in a place that {% data variables.product.prodname_importer_proper_name %} can access. This can be accomplished by:
|
|
|
|
|
|
|
|
|
|
* Using local storage on the GHES instance (GHES **3.16** and later)
|
|
|
|
|
* Using a blob storage provider
|
|
|
|
|
|
|
|
|
|
### Using local storage (GHES 3.16+)
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.local-storage-steps %}
|
|
|
|
|
|
|
|
|
|
### Using a blob storage provider
|
|
|
|
|
|
|
|
|
|
If your {% data variables.product.prodname_ghe_server %} instance is behind a firewall, you may need to set up blob storage with an external cloud service.
|
|
|
|
|
|
|
|
|
|
First, you must set up blob storage with a supported provider. Then, if you're using a cloud provider, you must configure your credentials for the storage provider in the {% data variables.enterprise.management_console %} or {% data variables.product.prodname_cli %}.
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.supported-blob-storage-providers %}
|
|
|
|
|
|
|
|
|
|
> [!NOTE]
|
|
|
|
|
> You only need to configure blob storage if you use {% data variables.product.prodname_ghe_server %} versions 3.8 or higher. If you use {% data variables.product.prodname_ghe_server %} versions 3.7 or lower, skip to [Step 4: Set up a migration source in {% data variables.product.prodname_ghe_cloud %}](#step-4-set-up-a-migration-source-in-github-enterprise-cloud).
|
|
|
|
|
>
|
|
|
|
|
> Blob storage is required to migrate repositories with large Git source or metadata. If you use {% data variables.product.prodname_ghe_server %} versions 3.7 or lower, you will not be able to perform migrations where your Git source or metadata exports exceed 2GB. To perform these migrations, update to {% data variables.product.prodname_ghe_server %} versions 3.8 or higher.
|
|
|
|
|
|
|
|
|
|
### Setting up an AWS S3 storage bucket
|
|
|
|
|
#### Setting up an AWS S3 storage bucket
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.set-up-aws-bucket %}
|
|
|
|
|
|
|
|
|
|
### Setting up an Azure Blob Storage account
|
|
|
|
|
#### Setting up an Azure Blob Storage account
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.set-up-azure-storage-account %}
|
|
|
|
|
|
|
|
|
|
### Using local storage (GHES 3.16+)
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.local-storage-steps %}
|
|
|
|
|
|
|
|
|
|
### Configuring blob storage in the {% data variables.enterprise.management_console %} of {% data variables.location.product_location_enterprise %}
|
|
|
|
|
#### Configuring blob storage in the {% data variables.enterprise.management_console %} of {% data variables.location.product_location_enterprise %}
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.blob-storage-management-console %}
|
|
|
|
|
|
|
|
|
|
### Allowing network access
|
|
|
|
|
#### Allowing network access
|
|
|
|
|
|
|
|
|
|
If you have configured firewall rules on your storage account, ensure you have allowed access to the IP ranges for your migration destination. See [AUTOTITLE](/migrations/using-github-enterprise-importer/migrating-between-github-products/managing-access-for-a-migration-between-github-products#configuring-ip-allow-lists-for-migrations).
|
|
|
|
|
|
|
|
|
|
@@ -239,6 +250,78 @@ If you're using {% data variables.product.prodname_ghe_server %} 3.8 or higher,
|
|
|
|
|
|
|
|
|
|
You may need to allowlist {% data variables.product.company_short %}'s IP ranges. For more information, see [AUTOTITLE](/migrations/using-github-enterprise-importer/migrating-between-github-products/managing-access-for-a-migration-between-github-products#configuring-ip-allow-lists-for-migrations).
|
|
|
|
|
|
|
|
|
|
### Uploading your migration archives to {% data variables.product.prodname_ghos %}
|
|
|
|
|
|
|
|
|
|
> [!NOTE]
|
|
|
|
|
> Repository migrations with {% data variables.product.prodname_ghos %} are currently in {% data variables.release-phases.public_preview %} and subject to change.
|
|
|
|
|
|
|
|
|
|
If you're using {% data variables.product.prodname_ghos %}, you will upload your archive to {% data variables.product.prodname_ghos %} using the following process:
|
|
|
|
|
|
|
|
|
|
1. Create a multipart upload by submitting a POST request
|
|
|
|
|
1. Upload archive in multiple parts up to 100MB in size as PATCH requests
|
|
|
|
|
1. Submit a PUT request to complete the upload
|
|
|
|
|
|
|
|
|
|
#### 1. Create the multipart upload
|
|
|
|
|
|
|
|
|
|
You will submit a POST request to:
|
|
|
|
|
|
|
|
|
|
```http
|
|
|
|
|
https://uploads.github.com/organizations/{organization_id}/gei/archive/blobs/uploads
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Include a JSON body like below with the archive name and size. The content type can remain as `"application/octet-stream"` for all uploads.
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"content_type": "application/octet-stream",
|
|
|
|
|
"name": "git-archive.tar.gz",
|
|
|
|
|
"size": 262144000
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This will return a JSON object response as follows:
|
|
|
|
|
|
|
|
|
|
```json
|
|
|
|
|
{
|
|
|
|
|
"guid": "363b2659-b8a3-4878-bfff-eed4bcb54d35",
|
|
|
|
|
"node_id": "MA_kgDaACQzNjNiMjY1OS1iOGEzLTQ4NzgtYmZmZi1lZWQ0YmNiNTRkMzU",
|
|
|
|
|
"name": "git-archive.tar.gz",
|
|
|
|
|
"size": 33287,
|
|
|
|
|
"uri": "gei://archive/363b2659-b8a3-4878-bfff-eed4bcb54d35",
|
|
|
|
|
"created_at": "2024-11-13T12:35:45.761-08:00"
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This URI represents the uploaded archive, and will be used to enqueue migration when you start your repository migration. The response will also include the location in the response header used to upload file parts using a PATCH request in the next step:
|
|
|
|
|
|
|
|
|
|
```http
|
|
|
|
|
/organizations/{organization_id}/gei/archive/blobs/uploads?part_number=1&guid=<guid>&upload_id=<upload_id>
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### 2. Upload the archive in multiple parts
|
|
|
|
|
|
|
|
|
|
Upload up to 100 MB of your file in each part with a PATCH request using the location from the previous response header, ensuring that the raw data is uploaded in the request body without using a multipart form. If the final part of your file is less than 100 MB, upload only the remaining bytes in that last request:
|
|
|
|
|
|
|
|
|
|
```http
|
|
|
|
|
https://uploads.github.com/{location}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This will return an empty response body with the following location in the response header:
|
|
|
|
|
|
|
|
|
|
```http
|
|
|
|
|
/organizations/{organization_id}/gei/archive/blobs/uploads?part_number=2&guid=<guid>&upload_id=<upload_id>
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Repeat this until the upload is complete. Ensure that you are reading up to 100 MB of the file at a time, and submitting requests to the new location values with the incremented `part_number` values.
|
|
|
|
|
|
|
|
|
|
#### 3. Complete the upload
|
|
|
|
|
|
|
|
|
|
Submit a PUT request to the "last path" value from the previous step with an empty body and your upload to GitHub-owned storage is complete. The GEI URI can be constructed with the GUID from this initial POST request in the following format:
|
|
|
|
|
|
|
|
|
|
```http
|
|
|
|
|
gei://archive/{guid}
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Step 7: Start your repository migration
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.start-repository-migration-ec %}
|
|
|
|
|
@@ -328,6 +411,19 @@ For {% data variables.product.pat_generic %} requirements, see [AUTOTITLE](/migr
|
|
|
|
|
|
|
|
|
|
## Step 4: Set up blob storage
|
|
|
|
|
|
|
|
|
|
### Migrating repositories with {% data variables.product.prodname_ghos %}
|
|
|
|
|
|
|
|
|
|
> [!NOTE]
|
|
|
|
|
> Repository migrations with {% data variables.product.prodname_ghos %} are currently in {% data variables.release-phases.public_preview %} and subject to change.
|
|
|
|
|
|
|
|
|
|
If you do not want to set up and provide {% data variables.product.prodname_importer_proper_name %} with access to a customer-owned blob storage account for storing your repository archives, you can migrate repositories using {% data variables.product.prodname_ghos %}. To do so, you must be running v1.9.0 (or higher) of {% data variables.product.prodname_gei_cli %}. {% data variables.product.prodname_ghos %} does not require additional setup and is available as an option when you run {% data variables.product.prodname_gei_cli %} commands.
|
|
|
|
|
|
|
|
|
|
For security purposes, {% data variables.product.prodname_ghos %} is explicitly write-only, and downloads from {% data variables.product.prodname_ghos %} are not possible. After a migration is complete, the repository archives are immediately deleted. If an archive is uploaded and not used in a migration, the archive is deleted after 7 days.
|
|
|
|
|
|
|
|
|
|
When you use the CLI flag for {% data variables.product.prodname_ghos %}, the repository archive is automatically exported to the destination configured in the Management Console, uploaded to GitHub-owned storage, and imported to your migration destination. When using {% data variables.product.prodname_ghos %} we recommend configuring local storage. See [Using local storage (GHES 3.16+)](#using-local-storage-ghes-316-1).
|
|
|
|
|
|
|
|
|
|
### Migrating repositories with customer-owned blob storage
|
|
|
|
|
|
|
|
|
|
{% data reusables.enterprise-migration-tool.blob-storage-intro %}
|
|
|
|
|
|
|
|
|
|
### Setting up an AWS S3 storage bucket
|
|
|
|
|
@@ -419,6 +515,7 @@ gh gei generate-script --github-source-org SOURCE \
|
|
|
|
|
| `--target-api-url TARGET-API-URL` | {% data reusables.enterprise-migration-tool.add-target-api-url %} |
|
|
|
|
|
| `--no-ssl-verify` | {% data reusables.enterprise-migration-tool.ssl-flag %} |
|
|
|
|
|
| `--download-migration-logs` | Download the migration log for each migrated repository. For more information about migration logs, see [AUTOTITLE](/migrations/using-github-enterprise-importer/completing-your-migration-with-github-enterprise-importer/accessing-your-migration-logs-for-github-enterprise-importer#downloading-all-migration-logs-for-an-organization). |
|
|
|
|
|
| `--use-github-storage`| Perform a repository migration using {% data variables.product.prodname_ghos %} as the intermediate blob storage solution. |
|
|
|
|
|
|
|
|
|
|
### Reviewing the migration script
|
|
|
|
|
|
|
|
|
|
@@ -495,6 +592,7 @@ gh gei migrate-repo --github-source-org SOURCE --source-repo CURRENT-NAME --gith
|
|
|
|
|
| `--no-ssl-verify` | {% data reusables.enterprise-migration-tool.ssl-flag %} |
|
|
|
|
|
| `--skip-releases` | {% data reusables.enterprise-migration-tool.skip-releases %} |
|
|
|
|
|
| `--target-repo-visibility TARGET-VISIBILITY` | {% data reusables.enterprise-migration-tool.set-repository-visibility %} |
|
|
|
|
|
| `--use-github-storage`| Perform a repository migration using {% data variables.product.prodname_ghos %} as the intermediate blob storage solution. |
|
|
|
|
|
|
|
|
|
|
#### Aborting the migration
|
|
|
|
|
|
|
|
|
|
|
Jim Boyle