Merge pull request #313 from erans/master

Forced setting a script execution path

.gitignore

@@ -17,3 +17,4 @@ redash/dump.rdb
 .ruby-version
 venv
 
+dump.rdb

.landscape.yaml

@@ -0,0 +1,2 @@
+ignore-paths:
+    - migrations

README.md

@@ -1,72 +1,46 @@
-# [_re:dash_](https://github.com/everythingme/redash)
-![Build Status](https://circleci.com/gh/EverythingMe/redash.png?circle-token=8a695aa5ec2cbfa89b48c275aea298318016f040 "Build Status")
+<p align="center">
+  <img title="re:dash" src='https://raw.githubusercontent.com/EverythingMe/redash/screenshots/redash_logo.png' />
+
+</p>
+<p align="center">
+    <img title="Build Status" src='https://circleci.com/gh/EverythingMe/redash.png?circle-token=8a695aa5ec2cbfa89b48c275aea298318016f040'/>
+</p>
 
 **_re:dash_** is our take on freeing the data within our company in a way that will better fit our culture and usage patterns.
 
-Prior to **_re:dash_**, we tried to use tranditional BI suites and discovered a set of bloated, technically challenged and slow tools/flows. What we were looking for was a more hacker'ish way to look at data, so we built one.
+Prior to **_re:dash_**, we tried to use traditional BI suites and discovered a set of bloated, technically challenged and slow tools/flows. What we were looking for was a more hacker'ish way to look at data, so we built one.
 
 **_re:dash_** was built to allow fast and easy access to billions of records, that we process and collect using Amazon Redshift ("petabyte scale data warehouse" that "speaks" PostgreSQL).
+Today **_re:dash_** has support for querying multiple databases, including: Redshift, Google BigQuery, PostgreSQL, MySQL, Graphite and custom scripts.
 
 **_re:dash_** consists of two parts:
 
-1. **Query Editor**: think of [JS Fiddle](http://jsfiddle.net) for SQL queries. It's your way to share data in the organization in an open way, by sharing both the dataset and the query that generated it. This way everyone can peer review not only the resulting dataset but also the process that generated it. Also it's possible to fork it and generate new datasets and reach new insights. 
-2. **Dashboards/Visualizations**: once you have a dataset, you can create different visualizations out of it, and then combine several visualizations into a single dashboard. Currently it supports bar charts, pivot table and cohorts.
+1. **Query Editor**: think of [JS Fiddle](http://jsfiddle.net) for SQL queries. It's your way to share data in the organization in an open way, by sharing both the dataset and the query that generated it. This way everyone can peer review not only the resulting dataset but also the process that generated it. Also it's possible to fork it and generate new datasets and reach new insights.
+2. **Dashboards/Visualizations**: once you have a dataset, you can create different visualizations out of it, and then combine several visualizations into a single dashboard. Currently it supports charts, pivot table and cohorts.
 
-This is the first release, which is more than usable but still has its rough edges and way to go to fulfill its full potential. The Query Editor part is quite solid, but the visualizations need more work to enrich them and to make them more user friendly.
+**_re:dash_** is a work in progress and has its rough edges and way to go to fulfill its full potential. The Query Editor part is quite solid, but the visualizations need more work to enrich them and to make them more user friendly.
 
 ## Demo
 
 ![Screenshots](https://raw.github.com/EverythingMe/redash/screenshots/screenshots.gif)
 
-You can try out the demo instance: http://rd-demo.herokuapp.com/ (login with any Google account).
+You can try out the demo instance: http://demo.redash.io/ (login with any Google account).
+
+## Getting Started
+
+* [Setting re:dash on your own server (Ubuntu)](https://github.com/EverythingMe/redash/wiki/Setting-re:dash-on-your-own-server-(for-version-0.4))
+* Additional documentation in the [Wiki](https://github.com/everythingme/redash/wiki).
 
-Due to Heroku dev plan limits, it has a small database of flights (see schema [here](http://rd-demo.herokuapp.com/dashboard/schema)). Also due to another Heroku limitation, it is running with the regular user, hence you can DELETE or INSERT data/tables. Please be nice and don't do this.
 
 ## Getting help
 
 * [Google Group (mailing list)](https://groups.google.com/forum/#!forum/redash-users): the best place to get updates about new releases or ask general questions.
-* #redash IRC channel on [Freenode](http://www.freenode.net/).
-
-## Technology
-
-* Python
-* [AngularJS](http://angularjs.org/)
-* [PostgreSQL](http://www.postgresql.org/) / [AWS Redshift](http://aws.amazon.com/redshift/)
-* [Redis](http://redis.io)
-
-PostgreSQL is used both as the operatinal database for the system, but also as the data store that is being queried. To be exact, we built this system to use on top of Amazon's Redshift, which supports the PG driver. But it's quite simple to add support for other datastores, and we do plan to do so.
-
-This is our first large scale AngularJS project, and we learned a lot during the development of it. There are still things we need to iron out, and comments on the way we use AngularJS are more than welcome (and pull requests just as well).
-
-### HighCharts
-
-HighCharts is really great, but it's not free for commercial use. Please refer to their [licensing options](http://shop.highsoft.com/highcharts.html), to see what applies for your use. 
-
-It's very likely that in the future we will switch to [D3.js](http://d3js.org/) instead.
-
-## Getting Started
-
-* [Setting up re:dash on Heroku in 5 minutes](https://github.com/EverythingMe/redash/wiki/Setting-up-re:dash-on-Heroku-in-5-minutes)
-* [Setting re:dash on your own server (Ubuntu)](https://github.com/EverythingMe/redash/wiki/Setting-re:dash-on-your-own-server-(Ubuntu))
-
-**Need help setting re:dash or one of the dependencies up?** Ping @arikfr on the IRC #redash channel or send a message to the [mailing list](https://groups.google.com/forum/#!forum/redash-users), and he will gladly help.
+* Find us [on gitter](https://gitter.im/EverythingMe/redash#) (chat).
+* Contact Arik, the maintainer directly: arik@everything.me.
 
 ## Roadmap
 
-Below you can see the "big" features of the next 3 releases (for full list, click on the link):
-
-### [v0.3](https://github.com/EverythingMe/redash/issues?milestone=2&state=open)
-
-- Dashboard filters: ability to filter/slice the data you see in a single dashboard using filters (date or selectors).
-- Multiple databases support (including other database type than PostgreSQL).
-- Scheduled reports by email.
-- Comments on queries.
-
-### [v0.4](https://github.com/EverythingMe/redash/issues?milestone=3&state=open)
-
-- Query versioning. 
-- More "realtime" UI (using websockets).
-- More visualizations.
+TBD.
 
 ## Reporting Bugs and Contributing Code
 

circle.yml

@@ -23,3 +23,6 @@ deployment:
     branch: master
     commands:
       - make upload
+notify:
+  webhooks:
+    - url: https://webhooks.gitter.im/e/895d09c3165a0913ac2f

manage.py

@@ -42,32 +42,40 @@ def check_settings():
 
 @manager.command
 def import_events(events_file):
+    # TODO: remove this code past 1/11/2014.
     import json
+    from collections import Counter
+
+    count = Counter()
 
-    count = 0
     with open(events_file) as f:
         for line in f:
             try:
                 event = json.loads(line)
 
-                user = event.pop('user_id')
-                action = event.pop('action')
-                object_type = event.pop('object_type')
-                object_id = event.pop('object_id')
-                created_at = datetime.datetime.utcfromtimestamp(event.pop('timestamp'))
-                additional_properties = json.dumps(event)
+                object_type = event.get('object_type', None)
+                object_id = event.get('object_id', None)
 
-                models.Event.create(user=user, action=action, object_type=object_type, object_id=object_id,
-                                    additional_properties=additional_properties, created_at=created_at)
+                if object_id == 'dashboard' and object_type == 'dashboard':
+                    count['bad dashboard id'] += 1
+                    continue
 
-                count += 1
+                models.Event.record(event)
+
+                count['imported'] += 1
 
             except Exception as ex:
                 print "Failed importing line:"
                 print line
                 print ex.message
+                count[ex.message] += 1
+                count['failed'] += 1
 
-    print "Importe %d rows" % count
+                models.db.close_db(None)
+
+    for k, v in count.iteritems():
+        print k
+        print v
 
 
 @database_manager.command

rd_ui/Gruntfile.js

@@ -187,7 +187,7 @@ module.exports = function (grunt) {
     // concat, minify and revision files. Creates configurations in memory so
     // additional tasks can operate on them
     useminPrepare: {
-      html: '<%= yeoman.app %>/index.html',
+      html: ['<%= yeoman.app %>/index.html', '<%= yeoman.app %>/login.html'],
       options: {
         dest: '<%= yeoman.dist %>',
         flow: {

rd_ui/app/index.html

@@ -15,6 +15,7 @@
     <link rel="stylesheet" href="/bower_components/pivottable/dist/pivot.css">
     <link rel="stylesheet" href="/bower_components/cornelius/src/cornelius.css">
     <link rel="stylesheet" href="/bower_components/select2/select2.css">
+    <link rel="stylesheet" href="/bower_components/angular-ui-select/dist/select.css">
     <link rel="stylesheet" href="/bower_components/pace/themes/pace-theme-minimal.css">
     <link rel="stylesheet" href="/styles/redash.css">
     <!-- endbuild -->
@@ -65,6 +66,12 @@
                 </ul>
             </li>
         </ul>
+        <form class="navbar-form navbar-left" role="search" ng-submit="searchQueries()">
+            <div class="form-group">
+                <input type="text" ng-model="term" class="form-control" placeholder="Search queries...">
+            </div>
+            <button type="submit" class="btn btn-default"><span class="glyphicon glyphicon-search"></span></button>
+        </form>
         <ul class="nav navbar-nav navbar-right">
             <p class="navbar-text avatar" ng-show="currentUser.id" ng-cloak>
                 <img ng-src="{{currentUser.gravatar_url}}" class="img-circle" alt="{{currentUser.name}}"/>
@@ -110,6 +117,7 @@
 <script src="/bower_components/mousetrap/plugins/global-bind/mousetrap-global-bind.js"></script>
 <script src="/bower_components/select2/select2.js"></script>
 <script src="/bower_components/angular-ui-select2/src/select2.js"></script>
+<script src="/bower_components/angular-ui-select/dist/select.js"></script>
 <script src="/bower_components/underscore.string/lib/underscore.string.js"></script>
 <script src="/bower_components/marked/lib/marked.js"></script>
 <script src="/scripts/ng_highchart.js"></script>
@@ -159,4 +167,4 @@
 </script>
 
 </body>
-</html>
+</html>

rd_ui/app/login.html

@@ -35,6 +35,19 @@
     <div class="row">
 
         <div class="main">
+            {% if show_google_openid %}
+
+            <div class="row">
+                <a href="/oauth/google?next={{next}}"><img src="/google_login.png" class="login-button"/></a>
+            </div>
+
+            <div class="login-or">
+                <hr class="hr-or">
+                <span class="span-or">or</span>
+            </div>
+
+            {% endif %}
+
             <form role="form" method="post" name="login">
                 <div class="form-group">
                     <label for="inputUsernameEmail">Username or email</label>
@@ -56,20 +69,7 @@
                 </button>
             </form>
 
-            {% if show_google_openid %}
 
-            <div class="login-or">
-                <hr class="hr-or">
-                <span class="span-or">or</span>
-            </div>
-
-            <div class="row">
-                <div class="col-xs-6 col-sm-6 col-md-6">
-                    <a href="/google_auth/login?next={{next}}" class="btn btn-lg btn-info btn-block">Google</a>
-                </div>
-            </div>
-
-            {% endif %}
         </div>
 
     </div>

rd_ui/app/scripts/app.js

@@ -14,7 +14,8 @@ angular.module('redash', [
     'ui.bootstrap',
     'smartTable.table',
     'ngResource',
-    'ngRoute'
+    'ngRoute',
+    'ui.select'
   ]).config(['$routeProvider', '$locationProvider', '$compileProvider', 'growlProvider',
     function ($routeProvider, $locationProvider, $compileProvider, growlProvider) {
       if (featureFlags.clientSideMetrics) {
@@ -37,7 +38,8 @@ angular.module('redash', [
 
       $routeProvider.when('/dashboard/:dashboardSlug', {
         templateUrl: '/views/dashboard.html',
-        controller: 'DashboardCtrl'
+        controller: 'DashboardCtrl',
+        reloadOnSearch: false
       });
       $routeProvider.when('/queries', {
         templateUrl: '/views/queries.html',
@@ -54,6 +56,11 @@ angular.module('redash', [
           }]
         }
       });
+      $routeProvider.when('/queries/search', {
+        templateUrl: '/views/queries_search_results.html',
+        controller: 'QuerySearchCtrl',
+        reloadOnSearch: true,
+      });
       $routeProvider.when('/queries/:queryId', {
         templateUrl: '/views/query.html',
         controller: 'QueryViewCtrl',

rd_ui/app/scripts/controllers/controllers.js

@@ -1,12 +1,71 @@
 (function () {
+  var QuerySearchCtrl = function($scope, $location, $filter, Events, Query) {
+    $scope.$parent.pageTitle = "Queries Search";
+
+    $scope.gridConfig = {
+      isPaginationEnabled: true,
+      itemsByPage: 50,
+      maxSize: 8,
+    };
+
+    var dateFormatter = function (value) {
+      if (!value) return "-";
+      return value.format("DD/MM/YY HH:mm");
+    }
+
+    $scope.gridColumns = [
+      {
+        "label": "Name",
+        "map": "name",
+        "cellTemplateUrl": "/views/queries_query_name_cell.html"
+      },
+      {
+        'label': 'Created By',
+        'map': 'user.name'
+      },
+      {
+        'label': 'Created At',
+        'map': 'created_at',
+        'formatFunction': dateFormatter
+      },
+      {
+        'label': 'Update Schedule',
+        'map': 'ttl',
+        'formatFunction': function (value) {
+          return $filter('refreshRateHumanize')(value);
+        }
+      }
+    ];
+
+    $scope.queries = [];
+    $scope.$parent.term = $location.search().q;
+
+    Query.search({q: $scope.term }, function(results) {
+      $scope.queries = _.map(results, function(query) {
+        query.created_at = moment(query.created_at);
+        return query;
+      });
+    });
+
+    $scope.search = function() {
+      if (!angular.isString($scope.term) || $scope.term.trim() == "") {
+        $scope.queries = [];
+        return;
+      }
+
+      $location.search({q: $scope.term});
+    };
+
+    Events.record(currentUser, "search", "query", "", {"term": $scope.term});
+  };
+
   var QueriesCtrl = function ($scope, $http, $location, $filter, Query) {
     $scope.$parent.pageTitle = "All Queries";
     $scope.gridConfig = {
       isPaginationEnabled: true,
       itemsByPage: 50,
       maxSize: 8,
-      isGlobalSearchActivated: true
-    }
+      isGlobalSearchActivated: true};
 
     $scope.allQueries = [];
     $scope.queries = [];
@@ -35,7 +94,7 @@
     Query.query(function (queries) {
       $scope.allQueries = _.map(queries, function (query) {
         query.created_at = moment(query.created_at);
-        query.last_retrieved_at = moment(query.last_retrieved_at);
+        query.retrieved_at = moment(query.retrieved_at);
         return query;
       });
 
@@ -58,35 +117,17 @@
         'formatFunction': dateFormatter
       },
       {
-        'label': 'Runtime (avg)',
-        'map': 'avg_runtime',
-        'formatFunction': function (value) {
-          return $filter('durationHumanize')(value);
-        }
-      },
-      {
-        'label': 'Runtime (min)',
-        'map': 'min_runtime',
-        'formatFunction': function (value) {
-          return $filter('durationHumanize')(value);
-        }
-      },
-      {
-        'label': 'Runtime (max)',
-        'map': 'max_runtime',
+        'label': 'Runtime',
+        'map': 'runtime',
         'formatFunction': function (value) {
           return $filter('durationHumanize')(value);
         }
       },
       {
         'label': 'Last Executed At',
-        'map': 'last_retrieved_at',
+        'map': 'retrieved_at',
         'formatFunction': dateFormatter
       },
-      {
-        'label': 'Times Executed',
-        'map': 'times_retrieved'
-      },
       {
         'label': 'Update Schedule',
         'map': 'ttl',
@@ -95,6 +136,7 @@
         }
       }
     ]
+
     $scope.tabs = [
       {"name": "My Queries", "key": "my"},
       {"key": "all", "name": "All Queries"},
@@ -110,7 +152,7 @@
     });
   }
 
-  var MainCtrl = function ($scope, Dashboard, notifications) {
+  var MainCtrl = function ($scope, $location, Dashboard, notifications) {
     if (featureFlags.clientSideMetrics) {
       $scope.$on('$locationChangeSuccess', function(event, newLocation, oldLocation) {
         // This will be called once per actual page load.
@@ -133,7 +175,11 @@
         $scope.otherDashboards = $scope.allDashboards['Other'] || [];
         $scope.groupedDashboards = _.omit($scope.allDashboards, 'Other');
       });
-    }
+    };
+
+    $scope.searchQueries = function() {
+      $location.path('/queries/search').search({q: $scope.term});
+    };
 
     $scope.reloadDashboards();
 
@@ -165,5 +211,6 @@
   angular.module('redash.controllers', [])
     .controller('QueriesCtrl', ['$scope', '$http', '$location', '$filter', 'Query', QueriesCtrl])
     .controller('IndexCtrl', ['$scope', 'Events', 'Dashboard', IndexCtrl])
-    .controller('MainCtrl', ['$scope', 'Dashboard', 'notifications', MainCtrl]);
+    .controller('MainCtrl', ['$scope', '$location', 'Dashboard', 'notifications', MainCtrl])
+    .controller('QuerySearchCtrl', ['$scope', '$location', '$filter', 'Events', 'Query',  QuerySearchCtrl]);
 })();

rd_ui/app/scripts/controllers/dashboard.js

@@ -1,5 +1,5 @@
 (function() {
-  var DashboardCtrl = function($scope, Events, Widget, $routeParams, $http, $timeout, Dashboard) {
+  var DashboardCtrl = function($scope, Events, Widget, $routeParams, $http, $timeout, $q, Dashboard) {
     $scope.refreshEnabled = false;
     $scope.refreshRate = 60;
 
@@ -8,43 +8,49 @@
         Events.record(currentUser, "view", "dashboard", dashboard.id);
 
         $scope.$parent.pageTitle = dashboard.name;
-        var filters = {};
+
+        var promises = [];
 
         $scope.dashboard.widgets = _.map($scope.dashboard.widgets, function (row) {
           return _.map(row, function (widget) {
             var w = new Widget(widget);
 
             if (w.visualization && dashboard.dashboard_filters_enabled) {
-              var queryFilters = w.getQuery().getQueryResult().getFilters();
-              _.each(queryFilters, function (filter) {
-                if (!_.has(filters, filter.name)) {
-                  // TODO: first object should be a copy, otherwise one of the chart filters behaves different than the others.
-                  filters[filter.name] = filter;
-                  filters[filter.name].originFilters = [];
-
-                  $scope.$watch(function () {
-                    return filter.current
-                  }, function (value) {
-                    _.each(filter.originFilters, function (originFilter) {
-                      originFilter.current = value;
-                    })
-                  });
-
-                }
-                ;
-
-                // TODO: merge values.
-                filters[filter.name].originFilters.push(filter);
-              });
+              promises.push(w.getQuery().getQueryResultPromise());
             }
 
             return w;
           });
         });
 
-        if (dashboard.dashboard_filters_enabled) {
-          $scope.filters = _.values(filters);
-        }
+        $q.all(promises).then(function(queryResults) {
+          var filters = {};
+          _.each(queryResults, function(queryResult) {
+            var queryFilters = queryResult.getFilters();
+            _.each(queryFilters, function (filter) {
+              if (!_.has(filters, filter.name)) {
+                // TODO: first object should be a copy, otherwise one of the chart filters behaves different than the others.
+                filters[filter.name] = filter;
+                filters[filter.name].originFilters = [];
+
+                $scope.$watch(function () { return filter.current }, function (value) {
+                  _.each(filter.originFilters, function (originFilter) {
+                    originFilter.current = value;
+                  });
+                });
+              };
+
+              // TODO: merge values.
+              filters[filter.name].originFilters.push(filter);
+            });
+          });
+
+          if (dashboard.dashboard_filters_enabled) {
+            $scope.filters = _.values(filters);
+          }
+        });
+
+
       }, function () {
         // error...
         // try again. we wrap loadDashboard with throttle so it doesn't happen too often.\
@@ -131,7 +137,7 @@
   };
 
   angular.module('redash.controllers')
-    .controller('DashboardCtrl', ['$scope', 'Events', 'Widget', '$routeParams', '$http', '$timeout', 'Dashboard', DashboardCtrl])
+    .controller('DashboardCtrl', ['$scope', 'Events', 'Widget', '$routeParams', '$http', '$timeout', '$q', 'Dashboard', DashboardCtrl])
     .controller('WidgetCtrl', ['$scope', 'Events', 'Query', WidgetCtrl])
 
 })();

rd_ui/app/scripts/directives/dashboard_directives.js

@@ -147,22 +147,22 @@
           var reset = function() {
             $scope.saveInProgress = false;
             $scope.widgetSize = 1;
-            $scope.queryId = null;
             $scope.selectedVis = null;
-            $scope.query = null;
+            $scope.query = {};
+            $scope.selected_query = undefined;
             $scope.text = "";
           };
 
           reset();
 
           $scope.loadVisualizations = function () {
-            if (!$scope.queryId) {
+            if (!$scope.query.selected) {
               return;
             }
 
-            Query.get({ id: $scope.queryId }, function(query) {
+            Query.get({ id: $scope.query.selected.id }, function(query) {
               if (query) {
-                $scope.query = query;
+                $scope.selected_query = query;
                 if (query.visualizations.length) {
                   $scope.selectedVis = query.visualizations[0];
                 }
@@ -170,6 +170,20 @@
             });
           };
 
+          $scope.searchQueries = function (term) {
+            if (!term || term.length < 3) {
+              return;
+            }
+
+            Query.search({q: term}, function(results) {
+              $scope.queries = results;
+            });
+          };
+
+          $scope.$watch('query', function () {
+            $scope.loadVisualizations();
+          }, true);
+
           $scope.saveWidget = function() {
             $scope.saveInProgress = true;
 

rd_ui/app/scripts/directives/directives.js

@@ -97,14 +97,24 @@
         value: '=',
         ignoreBlanks: '=',
         editable: '=',
-        done: '='
+        done: '=',
       },
       template: function (tElement, tAttrs) {
         var elType = tAttrs.editor || 'input';
         var placeholder = tAttrs.placeholder || 'Click to edit';
-        return '<span ng-click="editable && edit()" ng-bind="value" ng-class="{editable: editable}"></span>' +
-          '<span ng-click="editable && edit()" ng-show="editable && !value" ng-class="{editable: editable}">' + placeholder + '</span>' +
-          '<{elType} ng-model="value" class="rd-form-control"></{elType}>'.replace('{elType}', elType);
+
+        var viewMode = '';
+
+        if (tAttrs.markdown == "true") {
+          viewMode = '<span ng-click="editable && edit()" ng-bind-html="value|markdown" ng-class="{editable: editable}"></span>';
+        } else {
+          viewMode = '<span ng-click="editable && edit()" ng-bind="value" ng-class="{editable: editable}"></span>';
+        }
+
+        var placeholderSpan = '<span ng-click="editable && edit()" ng-show="editable && !value" ng-class="{editable: editable}">' + placeholder + '</span>';
+        var editor = '<{elType} ng-model="value" class="rd-form-control"></{elType}>'.replace('{elType}', elType);
+
+        return viewMode + placeholderSpan + editor;
       },
       link: function ($scope, element, attrs) {
         // Let's get a reference to the input element, as we'll want to reference it.
@@ -224,4 +234,17 @@
         '</span>'
     }
   });
+
+  // Used instead of autofocus attribute, which doesn't work in Angular as there is no real page load.
+  directives.directive('autofocus',
+    ['$timeout', function ($timeout) {
+      return {
+        link: function (scope, element) {
+          $timeout(function () {
+            element[0].focus();
+          });
+        }
+      };
+    }]
+  );
 })();

rd_ui/app/scripts/filters.js

@@ -70,6 +70,18 @@ angular.module('redash.filters', []).
 
   .filter('markdown', ['$sce', function ($sce) {
     return function (text) {
+      if (!text) {
+        return "";
+      }
       return $sce.trustAsHtml(marked(text));
     }
-  }]);
+  }])
+
+  .filter('trustAsHtml', ['$sce', function ($sce) {
+    return function (text) {
+      if (!text) {
+        return "";
+      }
+      return $sce.trustAsHtml(text);
+    }
+  }]);

rd_ui/app/scripts/services/resources.js

@@ -1,5 +1,5 @@
 (function () {
-  var QueryResult = function ($resource, $timeout) {
+  var QueryResult = function ($resource, $timeout, $q) {
     var QueryResultResource = $resource('/api/query_results/:id', {id: '@id'}, {'post': {'method': 'POST'}});
     var Job = $resource('/api/jobs/:id', {id: '@id'});
 
@@ -15,7 +15,11 @@
         _.each(this.query_result.data.rows, function (row) {
           _.each(row, function (v, k) {
             if (angular.isNumber(v)) {
-              columnTypes[k] = 'float';
+                if (parseInt(v) === v) {
+                    columnTypes[k] = 'integer';
+                } else {
+                    columnTypes[k] = 'float';
+                }
             } else if (_.isString(v) && v.match(/^\d{4}-\d{2}-\d{2}T/)) {
               row[k] = moment(v);
               columnTypes[k] = 'datetime';
@@ -32,6 +36,7 @@
           }
         });
 
+        this.deferred.resolve(this);
       } else if (this.job.status == 3) {
         this.status = "processing";
       } else {
@@ -40,6 +45,7 @@
     }
 
     function QueryResult(props) {
+      this.deferred = $q.defer();
       this.job = {};
       this.query_result = {};
       this.status = "waiting";
@@ -349,6 +355,10 @@
       });
 
       return queryResult;
+    };
+
+    QueryResult.prototype.toPromise = function() {
+      return this.deferred.promise;
     }
 
     QueryResult.get = function (data_source_id, query, ttl) {
@@ -369,7 +379,7 @@
   };
 
   var Query = function ($resource, QueryResult, DataSource) {
-    var Query = $resource('/api/queries/:id', {id: '@id'});
+    var Query = $resource('/api/queries/:id', {id: '@id'}, {search: {method: 'get', isArray: true, url: "/api/queries/search"}});
 
     Query.newQuery = function () {
       return new Query({
@@ -404,9 +414,15 @@
       return this.queryResult;
     };
 
+    Query.prototype.getQueryResultPromise = function() {
+      return this.getQueryResult().toPromise();
+    }
+
     return Query;
   };
 
+
+
   var DataSource = function ($resource) {
     var DataSourceResource = $resource('/api/data_sources/:id', {id: '@id'}, {'get': {'method': 'GET', 'cache': true, 'isArray': true}});
 
@@ -435,7 +451,7 @@
   }
 
   angular.module('redash.services')
-      .factory('QueryResult', ['$resource', '$timeout', QueryResult])
+      .factory('QueryResult', ['$resource', '$timeout', '$q', QueryResult])
       .factory('Query', ['$resource', 'QueryResult', 'DataSource', Query])
       .factory('DataSource', ['$resource', DataSource])
       .factory('Widget', ['$resource', 'Query', Widget]);

rd_ui/app/scripts/visualizations/base.js

@@ -55,7 +55,7 @@
     }];
   };
 
-  var VisualizationRenderer = function (Visualization) {
+  var VisualizationRenderer = function ($location, Visualization) {
     return {
       restrict: 'E',
       scope: {
@@ -70,10 +70,44 @@
       link: function (scope) {
         scope.select2Options = {
           width: '50%'
+        };
+
+        function readURL() {
+          var searchFilters = angular.fromJson($location.search().filters);
+          if (searchFilters) {
+            _.forEach(scope.filters, function(filter) {
+              var value = searchFilters[filter.friendlyName];
+              if (value) {
+                filter.current = value;
+              }
+            });
+          }
         }
+
+        function updateURL(filters) {
+          var current = {};
+          _.each(filters, function(filter) {
+            if (filter.current) {
+              current[filter.friendlyName] = filter.current;
+            }
+          });
+
+          var newSearch = angular.extend($location.search(), {
+            filters: angular.toJson(current)
+          });
+          $location.search(newSearch);
+        }
+
         scope.$watch('queryResult && queryResult.getFilters()', function (filters) {
           if (filters) {
             scope.filters = filters;
+
+            if (filters.length && false) {
+              readURL();
+
+              // start watching for changes and update URL
+              scope.$watch('filters', updateURL, true);
+            }
           }
         });
       }
@@ -172,7 +206,7 @@
 
   angular.module('redash.visualization', [])
       .provider('Visualization', VisualizationProvider)
-      .directive('visualizationRenderer', ['Visualization', VisualizationRenderer])
+      .directive('visualizationRenderer', ['$location', 'Visualization', VisualizationRenderer])
       .directive('visualizationOptionsEditor', ['Visualization', VisualizationOptionsEditor])
       .directive('filters', Filters)
       .directive('editVisulatizationForm', ['Events', 'Visualization', 'growl', EditVisualizationForm])

rd_ui/app/scripts/visualizations/chart.js

@@ -33,7 +33,7 @@
         $scope.chartSeries = [];
         $scope.chartOptions = {};
 
-        var reloadData = _.throttle(function(data) {
+        var reloadData = function(data) {
           if (!data || ($scope.queryResult && $scope.queryResult.getData()) == null) {
             $scope.chartSeries.splice(0, $scope.chartSeries.length);
           } else {
@@ -49,8 +49,8 @@
               }
               $scope.chartSeries.push(_.extend(s, additional));
             });
-          }
-        }, 500);
+          };
+        };
 
         $scope.$watch('options', function (chartOptions) {
           if (chartOptions) {
@@ -87,6 +87,8 @@
           'Pie': 'pie'
         };
 
+        scope.globalSeriesType = 'column';
+
         scope.stackingOptions = {
           "None": "none",
           "Normal": "normal",
@@ -120,6 +122,13 @@
         var chartOptionsUnwatch = null,
             columnsWatch = null;
 
+        scope.$watch('globalSeriesType', function(type, old) {
+          if (type && old && type !== old && scope.visualization.options.seriesOptions) {
+            _.each(scope.visualization.options.seriesOptions, function(sOptions) {
+              sOptions.type = type;
+            });
+          }
+        });
         scope.$watch('visualization.type', function (visualizationType) {
           if (visualizationType == 'CHART') {
             if (scope.visualization.options.series.stacking === null) {
@@ -135,7 +144,9 @@
 
               // TODO: remove uneeded ones?
               if (scope.visualization.options.seriesOptions == undefined) {
-                scope.visualization.options.seriesOptions = {};
+                scope.visualization.options.seriesOptions = {
+                  type: scope.globalSeriesType
+                };
               };
 
               _.each(scope.series, function(s, i) {
@@ -230,4 +241,4 @@
       }
     }
   });
-}());
+}());

rd_ui/app/styles/login.css

@@ -1,14 +1,15 @@
 .main {
     max-width: 320px;
     margin: 0 auto;
+    margin-top:20px;
 }
 
 .login-or {
     position: relative;
     font-size: 18px;
     color: #aaa;
-    margin-top: 10px;
-            margin-bottom: 10px;
+    margin-top: 20px;
+    margin-bottom: 20px;
     padding-top: 10px;
     padding-bottom: 10px;
 }
@@ -31,7 +32,9 @@
     margin-bottom: 0px !important;
 }
 
-/*h3 {*/
-    /*text-align: center;*/
-    /*line-height: 300%;*/
-/*}*/
+img.login-button {
+    width: 250px;
+    display: block;
+    margin-left: auto;
+    margin-right: auto;
+}

rd_ui/app/views/dashboard.html

@@ -29,7 +29,7 @@
                             <span ng-hide="currentUser.hasPermission('view_query')">{{query.name}}</span>
                             <query-link query="query" visualization="widget.visualization" ng-show="currentUser.hasPermission('view_query')"></query-link>
                         </p>
-                        <div class="text-muted" ng-bind="query.description"></div>
+                        <div class="text-muted" ng-bind-html="query.description | markdown"></div>
                     </h3>
                 </div>
 

rd_ui/app/views/new_widget_form.html

@@ -22,22 +22,22 @@
                 </div>
 
                 <div ng-show="isVisualization()">
-                    <p>
-                        <form class="form-inline" role="form" ng-submit="loadVisualizations()">
-                            <div class="form-group">
-                                <input class="form-control" placeholder="Query Id" ng-model="queryId">
-                            </div>
-                            <button type="submit" class="btn btn-primary" ng-disabled="!queryId">
-                                Load visualizations
-                            </button>
-                        </form>
-                    </p>
+                    <div class="form-group">
+                        <ui-select ng-model="query.selected" theme="bootstrap" reset-search-input="false">
+                            <ui-select-match placeholder="Search a query by name">{{$select.selected.name}}</ui-select-match>
+                            <ui-select-choices repeat="q in queries"
+                                               refresh="searchQueries($select.search)"
+                                               refresh-delay="0">
+                                <div ng-bind-html="q.name | highlight: $select.search | trustAsHtml"></div>
+                            </ui-select-choices>
+                        </ui-select>
+                    </div>
 
-                    <div ng-show="query">
-                      <div class="form-group">
-                          <label for="">Choose Visualization</label>
-                          <select ng-model="selectedVis" ng-options="vis as vis.name group by vis.type for vis in query.visualizations" class="form-control"></select>
-                      </div>
+                    <div ng-show="selected_query">
+                        <div class="form-group">
+                            <label for="">Choose Visualization</label>
+                            <select ng-model="selectedVis" ng-options="vis as vis.name group by vis.type for vis in selected_query.visualizations" class="form-control"></select>
+                        </div>
                     </div>
                 </div>
 

rd_ui/app/views/queries_search_results.html

@@ -0,0 +1,19 @@
+<div class="container">
+    <div class="row">
+        <p>
+            <form class="form-inline" role="form" ng-submit="search()">
+                <div class="form-group">
+                    <input class="form-control" placeholder="Search..." ng-model="term" autofocus>
+                </div>
+                <button type="submit" class="btn btn-primary">
+                    <span class="glyphicon glyphicon-search"></span>
+                </button>
+            </form>
+        </p>
+
+        <smart-table rows="queries" columns="gridColumns"
+                 config="gridConfig"
+                 class="table table-condensed table-hover"></smart-table>
+    </div>
+
+</div>

rd_ui/app/views/query.html

@@ -12,7 +12,14 @@
                     </h2>
                     <p>
                         <em>
-                            <edit-in-place editable="isQueryOwner" done="saveDescription" editor="textarea" placeholder="No description" ignore-blanks='false' value="query.description"></edit-in-place>
+                            <edit-in-place editable="isQueryOwner"
+                                           done="saveDescription"
+                                           editor="textarea"
+                                           placeholder="No description"
+                                           ignore-blanks='false'
+                                           value="query.description"
+                                           markdown="true">
+                             </edit-in-place>
                         </em>
                     </p>
                 </div>
@@ -171,7 +178,7 @@
                             <edit-visulatization-form visualization="vis" query="query" query-result="queryResult" ng-show="canEdit"></edit-visulatization-form>
                         </div>
 
-                        <div ng-show="selectedTab == 'add'">
+                        <div ng-if="canEdit" ng-show="selectedTab == 'add'">
                             <visualization-renderer visualization="newVisualization" query-result="queryResult"></visualization-renderer>
                             <edit-visulatization-form visualization="newVisualization" query="query" query-result="queryResult" ng-show="canEdit" open-editor="true" on-new-success="setVisualizationTab"></edit-visulatization-form>
                         </div>

rd_ui/app/views/visualizations/chart_editor.html

@@ -18,6 +18,15 @@
                             class="form-control"></select>
                 </div>
             </div>
+
+            <div class="form-group">
+                <label class="control-label col-sm-2">Series Type</label>
+
+                <div class="col-sm-10">
+                    <select required ng-options="value as key for (key, value) in seriesTypes"
+                            ng-model="globalSeriesType" class="form-control"></select>
+                </div>
+            </div>
         </div>
     </div>
 
@@ -88,4 +97,4 @@
 
         </div>
     </div>
-</div>
+</div>

rd_ui/app/views/visualizations/edit_visualization.html

@@ -24,4 +24,4 @@
     </div>
 
   </form>
-</div>
+</div>

rd_ui/bower.json

@@ -2,7 +2,10 @@
   "name": "rdUi",
   "version": "0.1.0",
   "dependencies": {
-    "angular": "1.2.7",
+    "angular": "1.2.18",
+    "angular-resource": "1.2.18",
+    "angular-route": "1.2.18",
+    "angular-growl": "0.4.0",
     "json3": "3.2.4",
     "jquery": "1.9.1",
     "bootstrap": "3.0.0",
@@ -13,9 +16,6 @@
     "angular-ui-codemirror": "0.0.5",
     "highcharts": "3.0.10",
     "underscore": "1.5.1",
-    "angular-resource": "1.2.15",
-    "angular-growl": "0.3.1",
-    "angular-route": "1.2.7",
     "pivottable": "~1.1.1",
     "cornelius": "https://github.com/restorando/cornelius.git",
     "gridster": "0.2.0",
@@ -25,13 +25,14 @@
     "underscore.string": "~2.3.3",
     "marked": "~0.3.2",
     "bucky": "~0.2.6",
-    "pace": "~0.5.1"
+    "pace": "~0.5.1",
+    "angular-ui-select": "0.8.2"
   },
   "devDependencies": {
-    "angular-mocks": "~1.0.7",
-    "angular-scenario": "~1.0.7"
+    "angular-mocks": "1.2.18",
+    "angular-scenario": "1.2.18"
   },
   "resolutions": {
-    "angular": "1.2.7"
+    "angular": "1.2.18"
   }
 }

rd_ui/package.json

@@ -29,7 +29,8 @@
     "karma-jasmine": "~0.1.5",
     "grunt-karma": "~0.8.3",
     "karma-phantomjs-launcher": "~0.1.4",
-    "karma": "~0.12.19"
+    "karma": "~0.12.19",
+    "karma-ng-html2js-preprocessor": "~0.1.0"
   },
   "engines": {
     "node": ">=0.10.0"

rd_ui/test/karma.conf.js

@@ -47,6 +47,7 @@ module.exports = function(config) {
       'app/bower_components/mousetrap/plugins/global-bind/mousetrap-global-bind.js',
       'app/bower_components/select2/select2.js',
       'app/bower_components/angular-ui-select2/src/select2.js',
+      'app/bower_components/angular-ui-select/dist/select.js',
       'app/bower_components/underscore.string/lib/underscore.string.js',
       'app/bower_components/marked/lib/marked.js',
       'app/scripts/ng_highchart.js',
@@ -75,10 +76,17 @@ module.exports = function(config) {
       'app/scripts/directives/dashboard_directives.js',
       'app/scripts/filters.js',
 
+      'app/views/**/*.html',
+
       'test/mocks/*.js',
       'test/unit/*.js'
     ],
 
+    // generate js files from html templates
+    preprocessors: {
+      'app/views/**/*.html': 'ng-html2js'
+    },
+
     // list of files / patterns to exclude
     exclude: [],
 
@@ -100,7 +108,8 @@ module.exports = function(config) {
     // Which plugins to enable
     plugins: [
       'karma-phantomjs-launcher',
-      'karma-jasmine'
+      'karma-jasmine',
+      'karma-ng-html2js-preprocessor'
     ],
 
     // Continuous Integration mode

rd_ui/test/mocks/redash_mocks.js

@@ -16,9 +16,8 @@ currentUser = {
 
 
 angular.module('redashMocks', [])
-  .value('mockData', {
+  .value('MockData', {
     query: {
-
       "ttl": -1,
       "query": "select name from users;",
       "id": 1803,
@@ -72,5 +71,38 @@ angular.module('redashMocks', [])
         "data_source_id": 1
       }
 
+    },
+
+    queryResult: {
+      "job": {},
+      "query_result": {
+        "retrieved_at": "2014-08-04T13:33:45.563486+03:00",
+        "query_hash": "9951c38c9cf00e6ee8aecce026b51c19",
+        "query": "select name as \"name::filter\" from users",
+        "runtime": 0.00896096229553223,
+        "data": {
+          "rows": [],
+          "columns": [{
+            "friendly_name": "name::filter",
+            "type": null,
+            "name": "name::filter"
+          }]
+        },
+        "id": 106673,
+        "data_source_id": 1
+      },
+      "status": "done",
+      "filters": [],
+      "filterFreeze": "test@example.com",
+      "updatedAt": "2014-08-05T13:13:40.833Z",
+      "columnNames": ["name::filter"],
+      "filteredData": [{
+        "name::filter": "test@example.com"
+      }],
+      "columns": [{
+        "friendly_name": "name::filter",
+        "type": null,
+        "name": "name::filter"
+      }]
     }
   });

rd_ui/test/unit/test_query_view.js

@@ -2,18 +2,18 @@
 
 describe('QueryViewCtrl', function() {
   var scope;
-  var mockData;
+  var MockData;
 
   beforeEach(module('redash', 'redashMocks'));
 
-  beforeEach(inject(function($injector, $controller, $rootScope, Query, _mockData_) {
-    mockData = _mockData_;
+  beforeEach(inject(function($injector, $controller, $rootScope, Query, _MockData_) {
+    MockData = _MockData_;
     scope = $rootScope.$new();
 
     var route = {
       current: {
         locals: {
-          query: new Query(mockData.query)
+          query: new Query(MockData.query)
         }
       }
     };

rd_ui/test/unit/test_visualization_renderer.js

@@ -0,0 +1,89 @@
+'use strict';
+
+describe('VisualizationRenderer', function() {
+  var element;
+  var scope;
+
+  var filters = [{
+    "name": "name::filter",
+    "friendlyName": "Name",
+    "values": ["test@example.com", "amirn@example.com"],
+    "multiple": false
+  }];
+
+  beforeEach(module('redash', 'redashMocks'));
+
+  // loading templates
+  beforeEach(module('app/views/grid_renderer.html',
+    'app/views/visualizations/filters.html'));
+
+  // serving templates
+  beforeEach(inject(function($httpBackend, $templateCache) {
+    $httpBackend.whenGET('/views/grid_renderer.html')
+      .respond($templateCache.get('app/views/grid_renderer.html'));
+
+    $httpBackend.whenGET('/views/visualizations/filters.html')
+      .respond($templateCache.get('app/views/visualizations/filters.html'));
+  }));
+
+  // directive setup
+  beforeEach(inject(function($rootScope, $compile, MockData, QueryResult) {
+    var qr = new QueryResult(MockData.queryResult)
+    qr.filters = filters;
+
+    $rootScope.queryResult = qr;
+
+    element = angular.element(
+      '<visualization-renderer query-result="queryResult">' +
+      '</visualization-renderer>');
+  }));
+
+
+  describe('scope', function() {
+    beforeEach(inject(function($rootScope, $compile) {
+      $compile(element)($rootScope);
+
+      // we will test the isolated scope of the directive
+      scope = element.isolateScope();
+      scope.$digest();
+    }));
+
+    it('should have filters', function() {
+      expect(scope.filters).toBeDefined();
+    });
+  });
+
+
+  /*describe('URL binding', function() {
+
+    beforeEach(inject(function($rootScope, $compile, $location) {
+      spyOn($location, 'search').andCallThrough();
+
+      // set initial search
+      var initialSearch = {};
+      initialSearch[filters[0].friendlyName] = filters[0].values[0];
+      $location.search('filters', initialSearch);
+
+      $compile(element)($rootScope);
+
+      // we will test the isolated scope of the directive
+      scope = element.isolateScope();
+      scope.$digest();
+    }));
+
+    it('should update scope from URL',
+      inject(function($location) {
+        expect($location.search).toHaveBeenCalled();
+        expect(scope.filters[0].current).toEqual(filters[0].values[0]);
+      }));
+
+    it('should update URL from scope',
+      inject(function($location) {
+        scope.filters[0].current = 'newValue';
+        scope.$digest();
+
+        var searchFilters = angular.fromJson($location.search().filters);
+        expect(searchFilters[filters[0].friendlyName]).toEqual('newValue');
+    }));
+  });*/
+});

redash/__init__.py

@@ -3,7 +3,7 @@ import urlparse
 import redis
 from statsd import StatsClient
 
-from redash import settings, events
+from redash import settings
 
 __version__ = '0.4.0'
 
@@ -15,8 +15,6 @@ def setup_logging():
     logging.getLogger().addHandler(handler)
     logging.getLogger().setLevel(settings.LOG_LEVEL)
 
-    events.setup_logging(settings.EVENTS_LOG_PATH, settings.EVENTS_CONSOLE_OUTPUT)
-
 
 def create_redis_connection():
     redis_url = urlparse.urlparse(settings.REDIS_URL)

redash/authentication.py

@@ -6,10 +6,8 @@ import logging
 
 from flask import request, make_response, redirect, url_for
 from flask.ext.login import LoginManager, login_user, current_user
-from flask.ext.googleauth import GoogleAuth, login
-from werkzeug.contrib.fixers import ProxyFix
 
-from redash import models, settings
+from redash import models, settings, google_oauth
 
 login_manager = LoginManager()
 logger = logging.getLogger('authentication')
@@ -57,48 +55,15 @@ class HMACAuthentication(object):
         return decorated
 
 
-def validate_email(email):
-    if not settings.GOOGLE_APPS_DOMAIN:
-        return True
-
-    return email in settings.ALLOWED_EXTERNAL_USERS or email.endswith("@%s" % settings.GOOGLE_APPS_DOMAIN)
-
-
-def create_and_login_user(app, user):
-    if not validate_email(user.email):
-        return
-
-    try:
-        user_object = models.User.get(models.User.email == user.email)
-        if user_object.name != user.name:
-            logger.debug("Updating user name (%r -> %r)", user_object.name, user.name)
-            user_object.name = user.name
-            user_object.save()
-    except models.User.DoesNotExist:
-        logger.debug("Creating user object (%r)", user.name)
-        user_object = models.User.create(name=user.name, email=user.email, groups = models.User.DEFAULT_GROUPS)
-
-    login_user(user_object, remember=True)
-
-login.connect(create_and_login_user)
-
-
 @login_manager.user_loader
 def load_user(user_id):
     return models.User.select().where(models.User.id == user_id).first()
 
 
 def setup_authentication(app):
-    if settings.GOOGLE_OPENID_ENABLED:
-        openid_auth = GoogleAuth(app, url_prefix="/google_auth")
-        # If we don't have a list of external users, we can use Google's federated login, which limits
-        # the domain with which you can sign in.
-        if not settings.ALLOWED_EXTERNAL_USERS and settings.GOOGLE_APPS_DOMAIN:
-            openid_auth._OPENID_ENDPOINT = "https://www.google.com/a/%s/o8/ud?be=o8" % settings.GOOGLE_APPS_DOMAIN
-
     login_manager.init_app(app)
     login_manager.anonymous_user = models.AnonymousUser
-    app.wsgi_app = ProxyFix(app.wsgi_app)
     app.secret_key = settings.COOKIE_SECRET
+    app.register_blueprint(google_oauth.blueprint)
 
     return HMACAuthentication()

redash/controllers.py

@@ -10,23 +10,20 @@ import json
 import numbers
 import cStringIO
 import datetime
+import logging
 
 from flask import render_template, send_from_directory, make_response, request, jsonify, redirect, \
     session, url_for
 from flask.ext.restful import Resource, abort
 from flask_login import current_user, login_user, logout_user
-
 import sqlparse
-import events
-from permissions import require_permission
 
 from redash import redis_connection, statsd_client, models, settings, utils, __version__
 from redash.wsgi import app, auth, api
+from redash.tasks import QueryTask, record_event
+from redash.cache import headers as cache_headers
+from redash.permissions import require_permission
 
-import logging
-from tasks import QueryTask
-
-from cache import headers as cache_headers
 
 @app.route('/ping', methods=['GET'])
 def ping():
@@ -69,8 +66,7 @@ def login():
         return redirect(request.args.get('next') or '/')
 
     if not settings.PASSWORD_LOGIN_ENABLED:
-        blueprint = app.extensions['googleauth'].blueprint
-        return redirect(url_for("%s.login" % blueprint.name, next=request.args.get('next')))
+        return redirect(url_for("google_oauth.authorize", next=request.args.get('next')))
 
     if request.method == 'POST':
         user = models.User.select().where(models.User.email == request.form['username']).first()
@@ -84,7 +80,7 @@ def login():
                            analytics=settings.ANALYTICS,
                            next=request.args.get('next'),
                            username=request.form.get('username', ''),
-                           show_google_openid=settings.GOOGLE_OPENID_ENABLED)
+                           show_google_openid=settings.GOOGLE_OAUTH_ENABLED)
 
 
 @app.route('/logout')
@@ -111,7 +107,6 @@ def status_api():
 
     manager_status = redis_connection.hgetall('redash:status')
     status['manager'] = manager_status
-    status['manager']['queue_size'] = redis_connection.llen('queries') + redis_connection.llen('scheduled_queries')
     status['manager']['outdated_queries_count'] = models.Query.outdated_queries().count()
 
     queues = {}
@@ -160,7 +155,7 @@ class EventAPI(BaseResource):
     def post(self):
         events_list = request.get_json(force=True)
         for event in events_list:
-            events.record_event(event)
+            record_event.delay(event)
 
 
 api.add_resource(EventAPI, '/api/events', endpoint='events')
@@ -281,6 +276,14 @@ api.add_resource(WidgetListAPI, '/api/widgets', endpoint='widgets')
 api.add_resource(WidgetAPI, '/api/widgets/<int:widget_id>', endpoint='widget')
 
 
+class QuerySearchAPI(BaseResource):
+    @require_permission('view_query')
+    def get(self):
+        term = request.args.get('q', '')
+
+        return [q.to_dict() for q in models.Query.search(term)]
+
+
 class QueryListAPI(BaseResource):
     @require_permission('create_query')
     def post(self):
@@ -329,6 +332,7 @@ class QueryAPI(BaseResource):
         else:
             abort(404, message="Query not found.")
 
+api.add_resource(QuerySearchAPI, '/api/queries/search', endpoint='queries_search')
 api.add_resource(QueryListAPI, '/api/queries', endpoint='queries')
 api.add_resource(QueryAPI, '/api/queries/<query_id>', endpoint='query')
 
@@ -414,48 +418,52 @@ class QueryResultListAPI(BaseResource):
 
 
 class QueryResultAPI(BaseResource):
-    @require_permission('view_query')
-    def get(self, query_result_id):
-        query_result = models.QueryResult.get_by_id(query_result_id)
-        if query_result:
-            data = json.dumps({'query_result': query_result.to_dict()}, cls=utils.JSONEncoder)
-            return make_response(data, 200, cache_headers)
-        else:
-            abort(404)
+    @staticmethod
+    def csv_response(query_result):
+        s = cStringIO.StringIO()
 
+        query_data = json.loads(query_result.data)
+        writer = csv.DictWriter(s, fieldnames=[col['name'] for col in query_data['columns']])
+        writer.writer = utils.UnicodeWriter(s)
+        writer.writeheader()
+        for row in query_data['rows']:
+            for k, v in row.iteritems():
+                if isinstance(v, numbers.Number) and (v > 1000 * 1000 * 1000 * 100):
+                    row[k] = datetime.datetime.fromtimestamp(v/1000.0)
+
+            writer.writerow(row)
+
+        headers = {'Content-Type': "text/csv; charset=UTF-8"}
+        headers.update(cache_headers)
+        return make_response(s.getvalue(), 200, headers)
 
-class CsvQueryResultsAPI(BaseResource):
     @require_permission('view_query')
-    def get(self, query_id, query_result_id=None):
-        if not query_result_id:
+    def get(self, query_id=None, query_result_id=None, filetype='json'):
+        if query_result_id is None and query_id is not None:
             query = models.Query.get(models.Query.id == query_id)
             if query:
                 query_result_id = query._data['latest_query_data']
 
-        query_result = query_result_id and models.QueryResult.get_by_id(query_result_id)
+        if query_result_id:
+            query_result = models.QueryResult.get_by_id(query_result_id)
+
         if query_result:
-            s = cStringIO.StringIO()
+            if filetype == 'json':
+                data = json.dumps({'query_result': query_result.to_dict()}, cls=utils.JSONEncoder)
+                return make_response(data, 200, cache_headers)
+            else:
+                return self.csv_response(query_result)
 
-            query_data = json.loads(query_result.data)
-            writer = csv.DictWriter(s, fieldnames=[col['name'] for col in query_data['columns']])
-            writer.writer = utils.UnicodeWriter(s)
-            writer.writeheader()
-            for row in query_data['rows']:
-                for k, v in row.iteritems():
-                    if isinstance(v, numbers.Number) and (v > 1000 * 1000 * 1000 * 100):
-                        row[k] = datetime.datetime.fromtimestamp(v/1000.0)
-
-                writer.writerow(row)
-
-            return make_response(s.getvalue(), 200, {'Content-Type': "text/csv; charset=UTF-8"})
         else:
             abort(404)
 
-api.add_resource(CsvQueryResultsAPI, '/api/queries/<query_id>/results/<query_result_id>.csv',
-                 '/api/queries/<query_id>/results.csv',
-                 endpoint='csv_query_results')
+
 api.add_resource(QueryResultListAPI, '/api/query_results', endpoint='query_results')
-api.add_resource(QueryResultAPI, '/api/query_results/<query_result_id>', endpoint='query_result')
+api.add_resource(QueryResultAPI,
+                 '/api/query_results/<query_result_id>',
+                 '/api/queries/<query_id>/results.<filetype>',
+                 '/api/queries/<query_id>/results/<query_result_id>.<filetype>',
+                 endpoint='query_result')
 
 
 class JobAPI(BaseResource):

redash/data/query_runner.py

@@ -23,8 +23,12 @@ def get_query_runner(connection_type, connection_string):
     elif connection_type == 'url':
         from redash.data import query_runner_url
         runner = query_runner_url.url(connection_string)
+    elif connection_type == "mongo":
+        from redash.data import query_runner_mongodb
+        connection_params = json.loads(connection_string)
+        runner = query_runner_mongodb.mongodb(connection_params)
     else:
         from redash.data import query_runner_pg
         runner = query_runner_pg.pg(connection_string)
 
-    return runner
+    return runner

redash/data/query_runner_mongodb.py

@@ -0,0 +1,150 @@
+import datetime
+import logging
+import json
+import sys
+import re
+import time
+from redash.utils import JSONEncoder
+
+try:
+    import pymongo
+    from bson.objectid import ObjectId
+except ImportError:
+    print "Missing dependencies. Please install pymongo."
+    print "You can use pip:   pip install pymongo"
+
+TYPES_MAP = {
+    ObjectId : "string",
+    str : "string",
+    unicode : "string",
+    int : "integer",
+    long : "integer",
+    float : "float",
+    bool : "boolean",
+    datetime.datetime: "datetime",
+}
+
+date_regex = re.compile("ISODate\(\"(.*)\"\)", re.IGNORECASE)
+
+def mongodb(connection_string):
+    def _get_column_by_name(columns, column_name):
+        for c in columns:
+            if "name" in c and c["name"] == column_name:
+                return c
+
+        return None
+
+    def _convert_date(q, field_name):
+        m = date_regex.findall(q[field_name])
+        if len(m) > 0:
+            if q[field_name].find(":") == -1:
+                q[field_name] = datetime.datetime.fromtimestamp(time.mktime(time.strptime(m[0], "%Y-%m-%d")))
+            else:
+                q[field_name] = datetime.datetime.fromtimestamp(time.mktime(time.strptime(m[0], "%Y-%m-%d %H:%M")))
+
+    def query_runner(query):
+        if not "dbName" in connection_string or not connection_string["dbName"]:
+            return None, "dbName is missing from connection string JSON or is empty"
+
+        db_name = connection_string["dbName"]
+
+        if not "connectionString" in connection_string or not connection_string["connectionString"]:
+            return None, "connectionString is missing from connection string JSON or is empty"
+
+        is_replica_set = True if "replicaSetName" in connection_string and connection_string["replicaSetName"] else False
+
+        if is_replica_set:
+            if not connection_string["replicaSetName"]:
+                return None, "replicaSetName is set in the connection string JSON but is empty"
+                
+            db_connection = pymongo.MongoReplicaSetClient(connection_string["connectionString"], replicaSet=connection_string["replicaSetName"])
+        else:
+            db_connection = pymongo.MongoClient(connection_string["connectionString"])
+
+        if db_name not in db_connection.database_names():
+            return None, "Unknown database name '%s'" % db_name
+
+        db = db_connection[db_name]
+
+        logging.debug("mongodb connection string: %s", connection_string)
+        logging.debug("mongodb got query: %s", query)
+
+        try:
+            query_data = json.loads(query)
+        except:
+            return None, "Invalid query format. The query is not a valid JSON."
+
+        collection = None
+        if not "collection" in query_data:
+            return None, "'collection' must have a value to run a query"
+        else:
+            collection = query_data["collection"]
+
+        q = None
+        if "query" in query_data:
+            q = query_data["query"]
+            for k in q:
+                if q[k] and type(q[k]) in [str, unicode]:
+                    logging.debug(q[k])
+                    _convert_date(q, k)
+                elif q[k] and type(q[k]) is dict:
+                    for k2 in q[k]:
+                        if type(q[k][k2]) in [str, unicode]:
+                            _convert_date(q[k], k2)
+
+        f = None
+        if "fields" in query_data:
+            f = query_data["fields"]
+
+        s = None
+        if "sort" in query_data and query_data["sort"]:
+            s = []
+            for field_name in query_data["sort"]:
+                s.append((field_name, query_data["sort"][field_name]))
+
+        columns = []
+        rows = []
+
+        error = None
+        json_data = None
+
+        cursor = None
+        if s:
+            cursor = db[collection].find(q, f).sort(s)
+        else:
+            cursor = db[collection].find(q, f)
+
+        for r in cursor:
+            for k in r:
+                if _get_column_by_name(columns, k) is None:
+                    columns.append({
+                        "name": k,
+                        "friendly_name": k,
+                        "type": TYPES_MAP[type(r[k])] if type(r[k]) in TYPES_MAP else None
+                    })
+
+                # Convert ObjectId to string
+                if type(r[k]) == ObjectId:
+                    r[k] = str(r[k])
+
+
+            rows.append(r)
+
+        if f:
+            ordered_columns = []
+            for k in sorted(f, key=f.get):
+                ordered_columns.append(_get_column_by_name(columns, k))
+
+            columns = ordered_columns
+
+        data = {
+            "columns": columns,
+            "rows": rows
+        }
+        error = None
+        json_data = json.dumps(data, cls=JSONEncoder)
+
+        return json_data, error
+
+    query_runner.annotate_query = False
+    return query_runner

redash/data/query_runner_mysql.py

@@ -18,7 +18,7 @@ def mysql(connection_string):
     
     def query_runner(query):
         connections_params = [entry.split('=')[1] for entry in connection_string.split(';')]
-        connection = MySQLdb.connect(*connections_params)
+        connection = MySQLdb.connect(*connections_params, charset="utf8", use_unicode=True)
         cursor = connection.cursor()
 
         logging.debug("mysql got query: %s", query)
@@ -61,4 +61,4 @@ def mysql(connection_string):
         return json_data, error
         
     
-    return query_runner
+    return query_runner

redash/data/query_runner_pg.py

@@ -88,11 +88,12 @@ def pg(connection_string):
             json_data = json.dumps(data, cls=JSONEncoder)
             error = None
             cursor.close()
-        except (select.error, OSError, psycopg2.OperationalError) as e:
+        except (select.error, OSError) as e:
             logging.exception(e)
             error = "Query interrupted. Please retry."
             json_data = None
         except psycopg2.DatabaseError as e:
+            logging.exception(e)
             json_data = None
             error = e.message
         except KeyboardInterrupt:

redash/data/query_runner_script.py

@@ -17,6 +17,9 @@ def script(connection_string):
             json_data = None
             error = None
 
+            if connection_string is None:
+                return None, "script execution path is not set. Please reconfigure the data source"
+
             # Poor man's protection against running scripts from output the scripts directory
             if connection_string.find("../") > -1:
                 return None, "Scripts can only be run from the configured scripts directory"

redash/events.py

@@ -1,23 +0,0 @@
-import logging
-import json
-
-logger = logging.getLogger("redash.events")
-logger.propagate = False
-
-
-def setup_logging(log_path, console_output=False):
-    if log_path:
-        fh = logging.FileHandler(log_path)
-        formatter = logging.Formatter('%(message)s')
-        fh.setFormatter(formatter)
-        logger.addHandler(fh)
-
-    if console_output:
-        handler = logging.StreamHandler()
-        formatter = logging.Formatter('[%(name)s] %(message)s')
-        handler.setFormatter(formatter)
-        logger.addHandler(handler)
-
-
-def record_event(event):
-    logger.info(json.dumps(event))

redash/google_oauth.py

@@ -0,0 +1,81 @@
+import logging
+from flask.ext.login import login_user
+import requests
+from flask import redirect, url_for, Blueprint
+from flask_oauth import OAuth
+from redash import models, settings
+
+logger = logging.getLogger('google_oauth')
+oauth = OAuth()
+
+request_token_params = {'scope': 'https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile', 'response_type': 'code'}
+
+if settings.GOOGLE_APPS_DOMAIN:
+    request_token_params['hd'] = settings.GOOGLE_APPS_DOMAIN
+else:
+    logger.warning("No Google Apps domain defined, all Google accounts allowed.")
+
+google = oauth.remote_app('google',
+                          base_url='https://www.google.com/accounts/',
+                          authorize_url='https://accounts.google.com/o/oauth2/auth',
+                          request_token_url=None,
+                          request_token_params=request_token_params,
+                          access_token_url='https://accounts.google.com/o/oauth2/token',
+                          access_token_method='POST',
+                          access_token_params={'grant_type': 'authorization_code'},
+                          consumer_key=settings.GOOGLE_CLIENT_ID,
+                          consumer_secret=settings.GOOGLE_CLIENT_SECRET)
+
+
+blueprint = Blueprint('google_oauth', __name__)
+
+
+def get_user_profile(access_token):
+    headers = {'Authorization': 'OAuth '+access_token}
+    response = requests.get('https://www.googleapis.com/oauth2/v1/userinfo', headers=headers)
+
+    if response.status_code == 401:
+        logger.warning("Failed getting user profile (response code 401).")
+        return None
+
+    return response.json()
+
+
+def create_and_login_user(name, email):
+    try:
+        user_object = models.User.get(models.User.email == email)
+        if user_object.name != name:
+            logger.debug("Updating user name (%r -> %r)", user_object.name, name)
+            user_object.name = name
+            user_object.save()
+    except models.User.DoesNotExist:
+        logger.debug("Creating user object (%r)", name)
+        user_object = models.User.create(name=name, email=email, groups=models.User.DEFAULT_GROUPS)
+
+    login_user(user_object, remember=True)
+
+
+@blueprint.route('/oauth/google', endpoint="authorize")
+def login():
+    # TODO, suport next
+    callback=url_for('.callback', _external=True)
+    logger.debug("Callback url: %s", callback)
+    return google.authorize(callback=callback)
+
+
+@blueprint.route('/oauth/google_callback', endpoint="callback")
+@google.authorized_handler
+def authorized(resp):
+    access_token = resp['access_token']
+
+    if access_token is None:
+        logger.warning("Access token missing in call back request.")
+        return redirect(url_for('login'))
+
+    profile = get_user_profile(access_token)
+    if profile is None:
+        return redirect(url_for('login'))
+
+    create_and_login_user(profile['name'], profile['email'])
+
+    return redirect(url_for('index'))

redash/models.py

@@ -305,11 +305,8 @@ class Query(BaseModel):
             d['user_id'] = self._data['user']
 
         if with_stats:
-            d['avg_runtime'] = self.avg_runtime
-            d['min_runtime'] = self.min_runtime
-            d['max_runtime'] = self.max_runtime
-            d['last_retrieved_at'] = self.last_retrieved_at
-            d['times_retrieved'] = self.times_retrieved
+            d['retrieved_at'] = self.retrieved_at
+            d['runtime'] = self.runtime
 
         if with_visualizations:
             d['visualizations'] = [vis.to_dict(with_query=False)
@@ -319,15 +316,10 @@ class Query(BaseModel):
 
     @classmethod
     def all_queries(cls):
-        q = Query.select(Query, User,
-                     peewee.fn.Count(QueryResult.id).alias('times_retrieved'),
-                     peewee.fn.Avg(QueryResult.runtime).alias('avg_runtime'),
-                     peewee.fn.Min(QueryResult.runtime).alias('min_runtime'),
-                     peewee.fn.Max(QueryResult.runtime).alias('max_runtime'),
-                     peewee.fn.Max(QueryResult.retrieved_at).alias('last_retrieved_at'))\
+        q = Query.select(Query, User, QueryResult.retrieved_at, QueryResult.runtime)\
             .join(QueryResult, join_type=peewee.JOIN_LEFT_OUTER)\
             .switch(Query).join(User)\
-            .group_by(Query.id, User.id)
+            .group_by(Query.id, User.id, QueryResult.id, QueryResult.retrieved_at, QueryResult.runtime)
 
         return q
 
@@ -348,6 +340,11 @@ class Query(BaseModel):
 
         return queries
 
+    @classmethod
+    def search(cls, term):
+        # This is very naive implementation of search, to be replaced with PostgreSQL full-text-search solution.
+        return cls.select().where((cls.name**"%{}%".format(term)) | (cls.description**"%{}%".format(term)))
+
     @classmethod
     def update_instance(cls, query_id, **kwargs):
         if 'query' in kwargs:
@@ -366,6 +363,14 @@ class Query(BaseModel):
             self.api_key = hashlib.sha1(
                 u''.join((str(time.time()), self.query, str(self._data['user']), self.name)).encode('utf-8')).hexdigest()
 
+    @property
+    def runtime(self):
+        return self.latest_query_data.runtime
+
+    @property
+    def retrieved_at(self):
+        return self.latest_query_data.retrieved_at
+
     def __unicode__(self):
         return unicode(self.id)
 
@@ -507,11 +512,10 @@ class Widget(BaseModel):
 
 
 class Event(BaseModel):
-    # user, action, object_type, object_id, additional_properties
     user = peewee.ForeignKeyField(User, related_name="events")
     action = peewee.CharField()
     object_type = peewee.CharField()
-    object_id = peewee.IntegerField()
+    object_id = peewee.CharField(null=True)
     additional_properties = peewee.TextField(null=True)
     created_at = peewee.DateTimeField(default=datetime.datetime.now)
 
@@ -521,6 +525,21 @@ class Event(BaseModel):
     def __unicode__(self):
         return u"%s,%s,%s,%s" % (self._data['user'], self.action, self.object_type, self.object_id)
 
+    @classmethod
+    def record(cls, event):
+        user = event.pop('user_id')
+        action = event.pop('action')
+        object_type = event.pop('object_type')
+        object_id = event.pop('object_id', None)
+
+        created_at = datetime.datetime.utcfromtimestamp(event.pop('timestamp'))
+        additional_properties = json.dumps(event)
+
+        event = cls.create(user=user, action=action, object_type=object_type, object_id=object_id,
+                           additional_properties=additional_properties, created_at=created_at)
+
+        return event
+
 
 all_models = (DataSource, User, QueryResult, Query, Dashboard, Visualization, Widget, ActivityLog, Group, Event)
 

redash/settings.py

@@ -59,18 +59,19 @@ CELERY_FLOWER_URL = os.environ.get("REDASH_CELERY_FLOWER_URL", "/flower")
 # Google Apps domain to allow access from; any user with email in this Google Apps will be allowed
 # access
 GOOGLE_APPS_DOMAIN = os.environ.get("REDASH_GOOGLE_APPS_DOMAIN", "")
-GOOGLE_OPENID_ENABLED = parse_boolean(os.environ.get("REDASH_GOOGLE_OPENID_ENABLED", "true"))
-PASSWORD_LOGIN_ENABLED = parse_boolean(os.environ.get("REDASH_PASSWORD_LOGIN_ENABLED", "false"))
-ALLOWED_EXTERNAL_USERS = array_from_string(os.environ.get("REDASH_ALLOWED_EXTERNAL_USERS", ''))
+
+GOOGLE_CLIENT_ID = os.environ.get("REDASH_GOOGLE_CLIENT_ID", "")
+GOOGLE_CLIENT_SECRET = os.environ.get("REDASH_GOOGLE_CLIENT_SECRET", "")
+GOOGLE_OAUTH_ENABLED = GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET
+
+PASSWORD_LOGIN_ENABLED = parse_boolean(os.environ.get("REDASH_PASSWORD_LOGIN_ENABLED", "true"))
 STATIC_ASSETS_PATH = fix_assets_path(os.environ.get("REDASH_STATIC_ASSETS_PATH", "../rd_ui/app/"))
 WORKERS_COUNT = int(os.environ.get("REDASH_WORKERS_COUNT", "2"))
-JOB_EXPIRY_TIME = int(os.environ.get("REDASH_JOB_EXPIRY_TIME", 3600*24))
+JOB_EXPIRY_TIME = int(os.environ.get("REDASH_JOB_EXPIRY_TIME", 3600*6))
 COOKIE_SECRET = os.environ.get("REDASH_COOKIE_SECRET", "c292a0a3aa32397cdb050e233733900f")
 LOG_LEVEL = os.environ.get("REDASH_LOG_LEVEL", "INFO")
-EVENTS_LOG_PATH = os.environ.get("REDASH_EVENTS_LOG_PATH", "")
-EVENTS_CONSOLE_OUTPUT = parse_boolean(os.environ.get("REDASH_EVENTS_CONSOLE_OUTPUT", "false"))
 CLIENT_SIDE_METRICS = parse_boolean(os.environ.get("REDASH_CLIENT_SIDE_METRICS", "false"))
 ANALYTICS = os.environ.get("REDASH_ANALYTICS", "")
 
 # Features:
-FEATURE_TABLES_PERMISSIONS = parse_boolean(os.environ.get("REDASH_FEATURE_TABLES_PERMISSIONS", "false"))
+FEATURE_TABLES_PERMISSIONS = parse_boolean(os.environ.get("REDASH_FEATURE_TABLES_PERMISSIONS", "false"))

redash/tasks.py

@@ -5,7 +5,7 @@ import redis
 from celery import Task
 from celery.result import AsyncResult
 from celery.utils.log import get_task_logger
-from redash import redis_connection, models, statsd_client
+from redash import redis_connection, models, statsd_client, settings
 from redash.utils import gen_query_hash
 from redash.worker import celery
 from redash.data.query_runner import get_query_runner
@@ -64,7 +64,12 @@ class QueryTask(object):
                     logging.info("[Manager][%s] Found existing job: %s", query_hash, job_id)
 
                     job = cls(job_id=job_id)
-                else:
+                    if job.ready():
+                        logging.info("[%s] job found is ready (%s), removing lock", query_hash, job.celery_status)
+                        redis_connection.delete(QueryTask._job_lock_id(query_hash, data_source.id))
+                        job = None
+
+                if not job:
                     pipe.multi()
 
                     if scheduled:
@@ -75,7 +80,7 @@ class QueryTask(object):
                     result = execute_query.apply_async(args=(query, data_source.id), queue=queue_name)
                     job = cls(async_result=result)
                     logging.info("[Manager][%s] Created new job: %s", query_hash, job.id)
-                    pipe.set(cls._job_lock_id(query_hash, data_source.id), job.id)
+                    pipe.set(cls._job_lock_id(query_hash, data_source.id), job.id, settings.JOB_EXPIRY_TIME)
                     pipe.execute()
                 break
 
@@ -113,6 +118,17 @@ class QueryTask(object):
             'query_result_id': query_result_id,
         }
 
+    @property
+    def is_cancelled(self):
+        return self._async_result.status == 'REVOKED'
+
+    @property
+    def celery_status(self):
+        return self._async_result.status
+
+    def ready(self):
+        return self._async_result.ready()
+
     def cancel(self):
         return self._async_result.revoke(terminate=True)
 
@@ -151,6 +167,41 @@ def refresh_queries():
     statsd_client.gauge('manager.seconds_since_refresh', now - float(status.get('last_refresh_at', now)))
 
 
+@celery.task(base=BaseTask)
+def cleanup_tasks():
+    # in case of cold restart of the workers, there might be jobs that still have their "lock" object, but aren't really
+    # going to run. this job removes them.
+
+    lock_keys = redis_connection.keys("query_hash_job:*") # TODO: use set instead of keys command
+    query_tasks = [QueryTask(job_id=j) for j in redis_connection.mget(lock_keys)]
+
+    logger.info("Found %d locks", len(query_tasks))
+
+    inspect = celery.control.inspect()
+    active_tasks = inspect.active()
+    if active_tasks is None:
+        active_tasks = []
+    else:
+        active_tasks = active_tasks.values()
+
+    all_tasks = set()
+    for task_list in active_tasks:
+        for task in task_list:
+            all_tasks.add(task['id'])
+
+    logger.info("Active jobs count: %d", len(all_tasks))
+
+    for i, t in enumerate(query_tasks):
+        if t.ready():
+            # if locked task is ready already (failed, finished, revoked), we don't need the lock anymore
+            logger.warning("%s is ready (%s), removing lock.", lock_keys[i], t.celery_status)
+            redis_connection.delete(lock_keys[i])
+
+        if t.celery_status == 'STARTED' and t.id not in all_tasks:
+            logger.warning("Couldn't find active job for: %s, removing lock.", lock_keys[i])
+            redis_connection.delete(lock_keys[i])
+
+
 @celery.task(bind=True, base=BaseTask, track_started=True)
 def execute_query(self, query, data_source_id):
     # TODO: maybe this should be a class?
@@ -195,3 +246,7 @@ def execute_query(self, query, data_source_id):
 
     return query_result.id
 
+
+@celery.task(base=BaseTask)
+def record_event(event):
+    models.Event.record(event)

redash/worker.py

@@ -13,6 +13,10 @@ celery.conf.update(CELERY_RESULT_BACKEND=settings.CELERY_BACKEND,
                            'task': 'redash.tasks.refresh_queries',
                            'schedule': timedelta(seconds=30)
                        },
+                       'cleanup_tasks': {
+                           'task': 'redash.tasks.cleanup_tasks',
+                           'schedule': timedelta(minutes=5)
+                       },
                    },
                    CELERY_TIMEZONE='UTC')
 

requirements.txt

@@ -1,7 +1,7 @@
 Flask==0.10.1
-Flask-GoogleAuth==0.4
 Flask-RESTful==0.2.10
 Flask-Login==0.2.9
+Flask-OAuth==0.12
 passlib==1.6.2
 Jinja2==2.7.2
 MarkupSafe==0.18
@@ -10,7 +10,7 @@ aniso8601==0.82
 blinker==1.3
 itsdangerous==0.23
 peewee==2.2.2
-psycopg2==2.5.1
+psycopg2==2.5.2
 python-dateutil==2.1
 pytz==2013.9
 redis==2.7.5

tests/test_authentication.py

@@ -1,52 +1,25 @@
-from unittest import TestCase
 from mock import patch
-from flask_googleauth import ObjectDict
 from tests import BaseTestCase
-from redash.authentication import validate_email, create_and_login_user
-from redash import settings, models
+from redash import models
+from redash.google_oauth import create_and_login_user
 from tests.factories import user_factory
 
 
-class TestEmailValidation(TestCase):
-    def test_accepts_address_with_correct_domain(self):
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', 'example.com'):
-            self.assertTrue(validate_email('example@example.com'))
-
-    def test_accepts_address_from_exception_list(self):
-        with patch.multiple(settings, GOOGLE_APPS_DOMAIN='example.com', ALLOWED_EXTERNAL_USERS=['whatever@whatever.com']):
-            self.assertTrue(validate_email('whatever@whatever.com'))
-
-    def test_accept_any_address_when_domain_empty(self):
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', None):
-            self.assertTrue(validate_email('whatever@whatever.com'))
-
-    def test_rejects_address_with_incorrect_domain(self):
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', 'example.com'):
-            self.assertFalse(validate_email('whatever@whatever.com'))
-
-
 class TestCreateAndLoginUser(BaseTestCase):
     def test_logins_valid_user(self):
         user = user_factory.create(email='test@example.com')
 
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', 'example.com'), patch('redash.authentication.login_user') as login_user_mock:
-            create_and_login_user(None, user)
+        with patch('redash.google_oauth.login_user') as login_user_mock:
+            create_and_login_user(user.name, user.email)
             login_user_mock.assert_called_once_with(user, remember=True)
 
     def test_creates_vaild_new_user(self):
-        openid_user = ObjectDict({'email': 'test@example.com', 'name': 'Test User'})
+        email = 'test@example.com'
+        name = 'Test User'
 
-        with patch.multiple(settings, GOOGLE_APPS_DOMAIN='example.com'), \
-             patch('redash.authentication.login_user') as login_user_mock:
+        with patch('redash.google_oauth.login_user') as login_user_mock:
 
-            create_and_login_user(None, openid_user)
+            create_and_login_user(name, email)
 
             self.assertTrue(login_user_mock.called)
-            user = models.User.get(models.User.email == openid_user.email)
-
-    def test_ignores_invliad_user(self):
-        user = ObjectDict({'email': 'test@whatever.com'})
-
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', 'example.com'), patch('redash.authentication.login_user') as login_user_mock:
-            create_and_login_user(None, user)
-            self.assertFalse(login_user_mock.called)
+            user = models.User.get(models.User.email == email)

tests/test_controllers.py

@@ -380,7 +380,7 @@ class TestLogin(BaseTestCase):
         with app.test_client() as c, patch.object(settings, 'PASSWORD_LOGIN_ENABLED', False):
             rv = c.get('/login')
             self.assertEquals(rv.status_code, 302)
-            self.assertTrue(rv.location.endswith(url_for('GoogleAuth.login')))
+            self.assertTrue(rv.location.endswith(url_for('google_oauth.authorize')))
 
     def test_get_login_form(self):
         with app.test_client() as c:

tests/test_models.py

@@ -1,7 +1,8 @@
 import datetime
+import json
 from tests import BaseTestCase
 from redash import models
-from factories import dashboard_factory, query_factory, data_source_factory, query_result_factory
+from factories import dashboard_factory, query_factory, data_source_factory, query_result_factory, user_factory
 from redash.utils import gen_query_hash
 
 
@@ -29,6 +30,28 @@ class QueryTest(BaseTestCase):
 
         self.assertNotEquals(old_hash, q.query_hash)
 
+    def test_search_finds_in_name(self):
+        q1 = query_factory.create(name="Testing search")
+        q2 = query_factory.create(name="Testing searching")
+        q3 = query_factory.create(name="Testing sea rch")
+
+        queries = models.Query.search("search")
+
+        self.assertIn(q1, queries)
+        self.assertIn(q2, queries)
+        self.assertNotIn(q3, queries)
+
+    def test_search_finds_in_description(self):
+        q1 = query_factory.create(description="Testing search")
+        q2 = query_factory.create(description="Testing searching")
+        q3 = query_factory.create(description="Testing sea rch")
+
+        queries = models.Query.search("search")
+
+        self.assertIn(q1, queries)
+        self.assertIn(q2, queries)
+        self.assertNotIn(q3, queries)
+
 
 class QueryResultTest(BaseTestCase):
     def setUp(self):
@@ -93,6 +116,7 @@ class QueryResultTest(BaseTestCase):
 
         self.assertEqual(found_query_result.id, qr.id)
 
+
 class TestQueryResultStoreResult(BaseTestCase):
     def setUp(self):
         super(TestQueryResultStoreResult, self).setUp()
@@ -148,4 +172,38 @@ class TestQueryResultStoreResult(BaseTestCase):
 
         self.assertEqual(models.Query.get_by_id(query1.id)._data['latest_query_data'], query_result.id)
         self.assertEqual(models.Query.get_by_id(query2.id)._data['latest_query_data'], query_result.id)
-        self.assertNotEqual(models.Query.get_by_id(query3.id)._data['latest_query_data'], query_result.id)
+        self.assertNotEqual(models.Query.get_by_id(query3.id)._data['latest_query_data'], query_result.id)
+
+
+class TestEvents(BaseTestCase):
+    def raw_event(self):
+        timestamp = 1411778709.791
+        user = user_factory.create()
+        created_at = datetime.datetime.utcfromtimestamp(timestamp)
+        raw_event = {"action": "view",
+                      "timestamp": timestamp,
+                      "object_type": "dashboard",
+                      "user_id": user.id,
+                      "object_id": 1}
+
+        return raw_event, user, created_at
+
+    def test_records_event(self):
+        raw_event, user, created_at = self.raw_event()
+
+        event = models.Event.record(raw_event)
+
+        self.assertEqual(event.user, user)
+        self.assertEqual(event.action, "view")
+        self.assertEqual(event.object_type, "dashboard")
+        self.assertEqual(event.object_id, 1)
+        self.assertEqual(event.created_at, created_at)
+
+    def test_records_additional_properties(self):
+        raw_event, _, _ = self.raw_event()
+        additional_properties = {'test': 1, 'test2': 2, 'whatever': "abc"}
+        raw_event.update(additional_properties)
+
+        event = models.Event.record(raw_event)
+
+        self.assertDictEqual(json.loads(event.additional_properties), additional_properties)