jprdonnelly/MachineLearningNotebooks
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Delete secrets.ipynb

Browse Source
This commit is contained in:
Shané Winner
2019-08-21 10:16:42 -07:00
committed by GitHub
parent 07a43a0444
commit 7620de2d91
1 changed files with 0 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files

140
work-with-data/dataprep/how-to-guides/secrets.ipynb
View File

@@ -1,140 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/work-with-data/dataprep/how-to-guides/secrets.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Providing Secrets\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Currently, secrets are only persisted for the lifetime of the engine process. Even if the dataflow is saved to a file, the secrets are not persisted in the dprep file. If you started a new session (i.e. start a new engine process), loaded a dataflow and wanted to run it, you will need to call `use_secrets` to register the required secrets to use during execution, otherwise the execution will fail as the required secrets are not available.\n",
"\n",
"In this notebook, we will:\n",
"1. Loading a previously saved dataflow\n",
"2. Call `get_missing_secrets` to determine the missing secrets\n",
"3. Call `use_secrets` and pass in the missing secrets to register it with the engine for this session\n",
"4. Call `head` to see the a preview of the data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import azureml.dataprep as dprep\n",
"\n",
"import os"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's load the previously saved dataflow."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dflow = dprep.Dataflow.open(file_path='../data/secrets.dprep')\n",
"dflow"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can call `get_missing_secrets` to see which required secrets are missing in the engine."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dflow.get_missing_secrets()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can now read the secrets from an environment variable, put it in a secret dictionary, and call `use_secrets` with the secrets. This will register the secrets in the engine so you don't need to provide them again in this session.\n",
"\n",
"_Note: It is a bad practice to have secrets in files that will be checked into source control._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"sas = os.environ['SCENARIOS_SECRETS']\n",
"secrets = {\n",
" 'https://dpreptestfiles.blob.core.windows.net/testfiles/read_csv_duplicate_headers.csv': sas\n",
"}\n",
"dflow.use_secrets(secrets=secrets)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can now call `head` without passing in `secrets` and the engine will successfully execute. Here is a preview of the data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dflow.head(5)"
]
}
],
"metadata": {
"authors": [
{
"name": "sihhu"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python36"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.4"
},
"notice": "Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License."
},
"nbformat": 4,
"nbformat_minor": 2
}