jprdonnelly/MachineLearningNotebooks
1
0
Fork 0
mirror of https://github.com/Azure/MachineLearningNotebooks.git synced 2025-12-21 01:55:07 -05:00
Code Issues Projects Releases Wiki Activity

Update notebooks

Browse Source
This commit is contained in:
Roope Astala
2018-10-12 14:39:33 -04:00
parent 216aa8b6a1
commit a4792d95ac
67 changed files with 6470 additions and 1610 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
automl/04.auto-ml-remote-execution-text-data-blob-store.ipynb
View File

@@ -132,15 +132,32 @@
"outputs": [],
"source": [
"from azureml.core.compute import RemoteCompute\n",
"import time\n",
"\n",
"# Add your VM information below\n",
"dsvm_name = 'mydsvm1'\n",
"# If a compute with the specified compute_name already exists, it will be used and the dsvm_ip_addr, dsvm_ssh_port, \n",
"# dsvm_username and dsvm_password will be ignored.\n",
"compute_name = 'mydsvm'\n",
"dsvm_ip_addr = '<<ip_addr>>'\n",
"dsvm_ssh_port = 22\n",
"dsvm_username = '<<username>>'\n",
"dsvm_password = '<<password>>'\n",
"\n",
"dsvm_compute = RemoteCompute.attach(workspace=ws, name=dsvm_name, address=dsvm_ip_addr, username=dsvm_username, password=dsvm_password, ssh_port=dsvm_ssh_port)"
"if compute_name in ws.compute_targets():\n",
" print('Using existing compute.')\n",
" dsvm_compute = ws.compute_targets()[compute_name]\n",
"else:\n",
" RemoteCompute.attach(workspace=ws, name=compute_name, address=dsvm_ip_addr, username=dsvm_username, password=dsvm_password, ssh_port=dsvm_ssh_port)\n",
"\n",
" while ws.compute_targets()[compute_name].provisioning_state == 'Creating':\n",
" time.sleep(1)\n",
"\n",
" dsvm_compute = ws.compute_targets()[compute_name]\n",
" \n",
" if dsvm_compute.provisioning_state == 'Failed':\n",
" print('Attached failed.')\n",
" print(dsvm_compute.provisioning_errors)\n",
" dsvm_compute.delete()"
]
},
{
@@ -182,11 +199,11 @@
" le = LabelEncoder()\n",
" le.fit(df[\"Label\"].values)\n",
" y = le.transform(df[\"Label\"].values)\n",
" df = df.drop([\"Label\"], axis=1)\n",
" X = df.drop([\"Label\"], axis=1)\n",
"\n",
" df_train, _, y_train, _ = train_test_split(df, y, test_size = 0.1, random_state = 42)\n",
" X_train, _, y_train, _ = train_test_split(X, y, test_size = 0.1, random_state = 42)\n",
"\n",
" return { \"X\" : df, \"y\" : y }"
" return { \"X\" : X_train, \"y\" : y_train }"
]
},
{
@@ -261,7 +278,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Train the Model <a class=\"anchor\" id=\"Training-the-model-Remote-DSVM\"></a>\n",
"## Train the Models <a class=\"anchor\" id=\"Training-the-model-Remote-DSVM\"></a>\n",
"\n",
"Call the `submit` method on the experiment object and pass the run configuration. For remote runs the execution is asynchronous, so you will see the iterations get populated as they complete. You can interact with the widgets and models even when the experiment is running to retrieve the best model up to that point. Once you are satisfied with the model, you can cancel a particular iteration or the whole run."
]
@@ -279,7 +296,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Exploring the results <a class=\"anchor\" id=\"Exploring-the-Results-Remote-DSVM\"></a>\n",
"## Exploring the Results <a class=\"anchor\" id=\"Exploring-the-Results-Remote-DSVM\"></a>\n",
"#### Widget for Monitoring Runs\n",
"\n",
"The widget will first report a \"loading\" status while running the first iteration. After completing the first iteration, an auto-updating graph and table will be shown. The widget will refresh once per minute, so you should see the graph update as child runs complete.\n",
@@ -329,7 +346,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Cancelling runs\n",
"## Cancelling Runs\n",
"You can cancel ongoing remote runs using the `cancel` and `cancel_iteration` functions."
]
},
@@ -352,7 +369,7 @@
"source": [
"### Retrieve the Best Model\n",
"\n",
"Below we select the best pipeline from our iterations. The `get_output` method on `automl_classifier` returns the best run and the fitted model for the last invocation. Overloads on `get_output` allow you to retrieve the best run and fitted model for *any* logged metric or for a particular *iteration*."
"Below we select the best pipeline from our iterations. The `get_output` method returns the best run and the fitted model. Overloads on `get_output` allow you to retrieve the best run and fitted model for *any* logged metric or for a particular *iteration*."
]
},
{
@@ -401,25 +418,6 @@
"zero_run, zero_model = remote_run.get_output(iteration = iteration)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Register the Fitted Model for Deployment"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"description = 'AutoML Model'\n",
"tags = None\n",
"remote_run.register_model(description = description, tags = tags)\n",
"print(remote_run.model_id) # Use this id to deploy the model as a web service in Azure."
]
},
{
"cell_type": "markdown",
"metadata": {},
@@ -445,12 +443,12 @@
"le = LabelEncoder()\n",
"le.fit(df[\"Label\"].values)\n",
"y = le.transform(df[\"Label\"].values)\n",
"df = df.drop([\"Label\"], axis=1)\n",
"X = df.drop([\"Label\"], axis=1)\n",
"\n",
"_, df_test, _, y_test = train_test_split(df, y, test_size=0.1, random_state=42)\n",
"_, X_test, _, y_test = train_test_split(X, y, test_size=0.1, random_state=42)\n",
"\n",
"\n",
"ypred = fitted_model.predict(df_test.values)\n",
"ypred = fitted_model.predict(X_test.values)\n",
"\n",
"\n",
"ypred_strings = le.inverse_transform(ypred)\n",
@@ -465,6 +463,11 @@
}
],
"metadata": {
"authors": [
{
"name": "savitam"
}
],
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",