Updated README: link to new demo instance.

Feature: basic search page for queries

README.md

@@ -1,72 +1,46 @@
-# [_re:dash_](https://github.com/everythingme/redash)
-![Build Status](https://circleci.com/gh/EverythingMe/redash.png?circle-token=8a695aa5ec2cbfa89b48c275aea298318016f040 "Build Status")
+<p align="center">
+  <img title="re:dash" src='https://raw.githubusercontent.com/EverythingMe/redash/screenshots/redash_logo.png' />
+
+</p>
+<p align="center">
+    <img title="Build Status" src='https://circleci.com/gh/EverythingMe/redash.png?circle-token=8a695aa5ec2cbfa89b48c275aea298318016f040'/>
+</p>
 
 **_re:dash_** is our take on freeing the data within our company in a way that will better fit our culture and usage patterns.
 
-Prior to **_re:dash_**, we tried to use tranditional BI suites and discovered a set of bloated, technically challenged and slow tools/flows. What we were looking for was a more hacker'ish way to look at data, so we built one.
+Prior to **_re:dash_**, we tried to use traditional BI suites and discovered a set of bloated, technically challenged and slow tools/flows. What we were looking for was a more hacker'ish way to look at data, so we built one.
 
 **_re:dash_** was built to allow fast and easy access to billions of records, that we process and collect using Amazon Redshift ("petabyte scale data warehouse" that "speaks" PostgreSQL).
+Today **_re:dash_** has support for querying multiple databases, including: Redshift, Google BigQuery, PostgreSQL, MySQL, Graphite and custom scripts.
 
 **_re:dash_** consists of two parts:
 
-1. **Query Editor**: think of [JS Fiddle](http://jsfiddle.net) for SQL queries. It's your way to share data in the organization in an open way, by sharing both the dataset and the query that generated it. This way everyone can peer review not only the resulting dataset but also the process that generated it. Also it's possible to fork it and generate new datasets and reach new insights. 
-2. **Dashboards/Visualizations**: once you have a dataset, you can create different visualizations out of it, and then combine several visualizations into a single dashboard. Currently it supports bar charts, pivot table and cohorts.
+1. **Query Editor**: think of [JS Fiddle](http://jsfiddle.net) for SQL queries. It's your way to share data in the organization in an open way, by sharing both the dataset and the query that generated it. This way everyone can peer review not only the resulting dataset but also the process that generated it. Also it's possible to fork it and generate new datasets and reach new insights.
+2. **Dashboards/Visualizations**: once you have a dataset, you can create different visualizations out of it, and then combine several visualizations into a single dashboard. Currently it supports charts, pivot table and cohorts.
 
-This is the first release, which is more than usable but still has its rough edges and way to go to fulfill its full potential. The Query Editor part is quite solid, but the visualizations need more work to enrich them and to make them more user friendly.
+**_re:dash_** is a work in progress and has its rough edges and way to go to fulfill its full potential. The Query Editor part is quite solid, but the visualizations need more work to enrich them and to make them more user friendly.
 
 ## Demo
 
 ![Screenshots](https://raw.github.com/EverythingMe/redash/screenshots/screenshots.gif)
 
-You can try out the demo instance: http://rd-demo.herokuapp.com/ (login with any Google account).
+You can try out the demo instance: http://demo.redash.io/ (login with any Google account).
+
+## Getting Started
+
+* [Setting re:dash on your own server (Ubuntu)](https://github.com/EverythingMe/redash/wiki/Setting-re:dash-on-your-own-server-(for-version-0.4))
+* Additional documentation in the [Wiki](https://github.com/everythingme/redash/wiki).
 
-Due to Heroku dev plan limits, it has a small database of flights (see schema [here](http://rd-demo.herokuapp.com/dashboard/schema)). Also due to another Heroku limitation, it is running with the regular user, hence you can DELETE or INSERT data/tables. Please be nice and don't do this.
 
 ## Getting help
 
 * [Google Group (mailing list)](https://groups.google.com/forum/#!forum/redash-users): the best place to get updates about new releases or ask general questions.
-* #redash IRC channel on [Freenode](http://www.freenode.net/).
-
-## Technology
-
-* Python
-* [AngularJS](http://angularjs.org/)
-* [PostgreSQL](http://www.postgresql.org/) / [AWS Redshift](http://aws.amazon.com/redshift/)
-* [Redis](http://redis.io)
-
-PostgreSQL is used both as the operatinal database for the system, but also as the data store that is being queried. To be exact, we built this system to use on top of Amazon's Redshift, which supports the PG driver. But it's quite simple to add support for other datastores, and we do plan to do so.
-
-This is our first large scale AngularJS project, and we learned a lot during the development of it. There are still things we need to iron out, and comments on the way we use AngularJS are more than welcome (and pull requests just as well).
-
-### HighCharts
-
-HighCharts is really great, but it's not free for commercial use. Please refer to their [licensing options](http://shop.highsoft.com/highcharts.html), to see what applies for your use. 
-
-It's very likely that in the future we will switch to [D3.js](http://d3js.org/) instead.
-
-## Getting Started
-
-* [Setting up re:dash on Heroku in 5 minutes](https://github.com/EverythingMe/redash/wiki/Setting-up-re:dash-on-Heroku-in-5-minutes)
-* [Setting re:dash on your own server (Ubuntu)](https://github.com/EverythingMe/redash/wiki/Setting-re:dash-on-your-own-server-(Ubuntu))
-
-**Need help setting re:dash or one of the dependencies up?** Ping @arikfr on the IRC #redash channel or send a message to the [mailing list](https://groups.google.com/forum/#!forum/redash-users), and he will gladly help.
+* Find us [on gitter](https://gitter.im/EverythingMe/redash#) (chat).
+* Contact Arik, the maintainer directly: arik@everything.me.
 
 ## Roadmap
 
-Below you can see the "big" features of the next 3 releases (for full list, click on the link):
-
-### [v0.3](https://github.com/EverythingMe/redash/issues?milestone=2&state=open)
-
-- Dashboard filters: ability to filter/slice the data you see in a single dashboard using filters (date or selectors).
-- Multiple databases support (including other database type than PostgreSQL).
-- Scheduled reports by email.
-- Comments on queries.
-
-### [v0.4](https://github.com/EverythingMe/redash/issues?milestone=3&state=open)
-
-- Query versioning. 
-- More "realtime" UI (using websockets).
-- More visualizations.
+TBD.
 
 ## Reporting Bugs and Contributing Code
 

circle.yml

@@ -23,3 +23,6 @@ deployment:
     branch: master
     commands:
       - make upload
+notify:
+  webhooks:
+    - url: https://webhooks.gitter.im/e/895d09c3165a0913ac2f

manage.py

@@ -42,6 +42,7 @@ def check_settings():
 
 @manager.command
 def import_events(events_file):
+    # TODO: remove this code past 1/11/2014.
     import json
     from collections import Counter
 
@@ -52,20 +53,14 @@ def import_events(events_file):
             try:
                 event = json.loads(line)
 
-                user = event.pop('user_id')
-                action = event.pop('action')
-                object_type = event.pop('object_type')
-                object_id = event.pop('object_id', None)
+                object_type = event.get('object_type', None)
+                object_id = event.get('object_id', None)
 
                 if object_id == 'dashboard' and object_type == 'dashboard':
                     count['bad dashboard id'] += 1
                     continue
 
-                created_at = datetime.datetime.utcfromtimestamp(event.pop('timestamp'))
-                additional_properties = json.dumps(event)
-
-                models.Event.create(user=user, action=action, object_type=object_type, object_id=object_id,
-                                    additional_properties=additional_properties, created_at=created_at)
+                models.Event.record(event)
 
                 count['imported'] += 1
 

rd_ui/app/index.html

@@ -65,6 +65,12 @@
                 </ul>
             </li>
         </ul>
+        <form class="navbar-form navbar-left" role="search" ng-submit="searchQueries()">
+            <div class="form-group">
+                <input type="text" ng-model="term" class="form-control" placeholder="Search queries...">
+            </div>
+            <button type="submit" class="btn btn-default"><span class="glyphicon glyphicon-search"></span></button>
+        </form>
         <ul class="nav navbar-nav navbar-right">
             <p class="navbar-text avatar" ng-show="currentUser.id" ng-cloak>
                 <img ng-src="{{currentUser.gravatar_url}}" class="img-circle" alt="{{currentUser.name}}"/>

rd_ui/app/login.html

@@ -35,6 +35,19 @@
     <div class="row">
 
         <div class="main">
+            {% if show_google_openid %}
+
+            <div class="row">
+                <a href="/oauth/google?next={{next}}"><img src="/google_login.png" class="login-button"/></a>
+            </div>
+
+            <div class="login-or">
+                <hr class="hr-or">
+                <span class="span-or">or</span>
+            </div>
+
+            {% endif %}
+
             <form role="form" method="post" name="login">
                 <div class="form-group">
                     <label for="inputUsernameEmail">Username or email</label>
@@ -56,20 +69,7 @@
                 </button>
             </form>
 
-            {% if show_google_openid %}
 
-            <div class="login-or">
-                <hr class="hr-or">
-                <span class="span-or">or</span>
-            </div>
-
-            <div class="row">
-                <div class="col-xs-6 col-sm-6 col-md-6">
-                    <a href="/google_auth/login?next={{next}}" class="btn btn-lg btn-info btn-block">Google</a>
-                </div>
-            </div>
-
-            {% endif %}
         </div>
 
     </div>

rd_ui/app/scripts/app.js

@@ -55,6 +55,11 @@ angular.module('redash', [
           }]
         }
       });
+      $routeProvider.when('/queries/search', {
+        templateUrl: '/views/queries_search_results.html',
+        controller: 'QuerySearchCtrl',
+        reloadOnSearch: true,
+      });
       $routeProvider.when('/queries/:queryId', {
         templateUrl: '/views/query.html',
         controller: 'QueryViewCtrl',

rd_ui/app/scripts/controllers/controllers.js

@@ -1,12 +1,63 @@
 (function () {
+  var QuerySearchCtrl = function($scope, $location, Query) {
+    $scope.$parent.pageTitle = "Queries Search";
+
+    $scope.gridConfig = {
+      isPaginationEnabled: true,
+      itemsByPage: 50,
+      maxSize: 8,
+    };
+
+    var dateFormatter = function (value) {
+      if (!value) return "-";
+      return value.format("DD/MM/YY HH:mm");
+    }
+
+    $scope.gridColumns = [
+      {
+        "label": "Name",
+        "map": "name",
+        "cellTemplateUrl": "/views/queries_query_name_cell.html"
+      },
+      {
+        'label': 'Created By',
+        'map': 'user.name'
+      },
+      {
+        'label': 'Created At',
+        'map': 'created_at',
+        'formatFunction': dateFormatter
+      }
+    ];
+
+    $scope.queries = [];
+    $scope.$parent.term = $location.search().q;
+
+    Query.search({q: $scope.term }, function(results) {
+      console.log(results);
+      $scope.queries = _.map(results, function(query) {
+        query.created_at = moment(query.created_at);
+        return query;
+      });
+    });
+
+    $scope.search = function() {
+      if (!angular.isString($scope.term) || $scope.term.trim() == "") {
+        $scope.queries = [];
+        return;
+      }
+
+      $location.search({q: $scope.term});
+    };
+  };
+
   var QueriesCtrl = function ($scope, $http, $location, $filter, Query) {
     $scope.$parent.pageTitle = "All Queries";
     $scope.gridConfig = {
       isPaginationEnabled: true,
       itemsByPage: 50,
       maxSize: 8,
-      isGlobalSearchActivated: true
-    }
+      isGlobalSearchActivated: true};
 
     $scope.allQueries = [];
     $scope.queries = [];
@@ -110,7 +161,7 @@
     });
   }
 
-  var MainCtrl = function ($scope, Dashboard, notifications) {
+  var MainCtrl = function ($scope, $location, Dashboard, notifications) {
     if (featureFlags.clientSideMetrics) {
       $scope.$on('$locationChangeSuccess', function(event, newLocation, oldLocation) {
         // This will be called once per actual page load.
@@ -133,7 +184,12 @@
         $scope.otherDashboards = $scope.allDashboards['Other'] || [];
         $scope.groupedDashboards = _.omit($scope.allDashboards, 'Other');
       });
-    }
+    };
+
+    $scope.searchQueries = function() {
+      console.log("search");
+      $location.path('/queries/search').search({q: $scope.term});
+    };
 
     $scope.reloadDashboards();
 
@@ -165,5 +221,6 @@
   angular.module('redash.controllers', [])
     .controller('QueriesCtrl', ['$scope', '$http', '$location', '$filter', 'Query', QueriesCtrl])
     .controller('IndexCtrl', ['$scope', 'Events', 'Dashboard', IndexCtrl])
-    .controller('MainCtrl', ['$scope', 'Dashboard', 'notifications', MainCtrl]);
+    .controller('MainCtrl', ['$scope', '$location', 'Dashboard', 'notifications', MainCtrl])
+    .controller('QuerySearchCtrl', ['$scope', '$location', 'Query', QuerySearchCtrl]);
 })();

rd_ui/app/scripts/directives/directives.js

@@ -97,14 +97,24 @@
         value: '=',
         ignoreBlanks: '=',
         editable: '=',
-        done: '='
+        done: '=',
       },
       template: function (tElement, tAttrs) {
         var elType = tAttrs.editor || 'input';
         var placeholder = tAttrs.placeholder || 'Click to edit';
-        return '<span ng-click="editable && edit()" ng-bind="value" ng-class="{editable: editable}"></span>' +
-          '<span ng-click="editable && edit()" ng-show="editable && !value" ng-class="{editable: editable}">' + placeholder + '</span>' +
-          '<{elType} ng-model="value" class="rd-form-control"></{elType}>'.replace('{elType}', elType);
+
+        var viewMode = '';
+
+        if (tAttrs.markdown == "true") {
+          viewMode = '<span ng-click="editable && edit()" ng-bind-html="value|markdown" ng-class="{editable: editable}"></span>';
+        } else {
+          viewMode = '<span ng-click="editable && edit()" ng-bind="value" ng-class="{editable: editable}"></span>';
+        }
+
+        var placeholderSpan = '<span ng-click="editable && edit()" ng-show="editable && !value" ng-class="{editable: editable}">' + placeholder + '</span>';
+        var editor = '<{elType} ng-model="value" class="rd-form-control"></{elType}>'.replace('{elType}', elType);
+
+        return viewMode + placeholderSpan + editor;
       },
       link: function ($scope, element, attrs) {
         // Let's get a reference to the input element, as we'll want to reference it.
@@ -224,4 +234,17 @@
         '</span>'
     }
   });
+
+  // Used instead of autofocus attribute, which doesn't work in Angular as there is no real page load.
+  directives.directive('autofocus',
+    ['$timeout', function ($timeout) {
+      return {
+        link: function (scope, element) {
+          $timeout(function () {
+            element[0].focus();
+          });
+        }
+      };
+    }]
+  );
 })();

rd_ui/app/scripts/services/resources.js

@@ -375,7 +375,7 @@
   };
 
   var Query = function ($resource, QueryResult, DataSource) {
-    var Query = $resource('/api/queries/:id', {id: '@id'});
+    var Query = $resource('/api/queries/:id', {id: '@id'}, {search: {method: 'get', isArray: true, url: "/api/queries/search"}});
 
     Query.newQuery = function () {
       return new Query({

rd_ui/app/scripts/visualizations/base.js

@@ -68,7 +68,6 @@
       template: '<filters></filters>\n' + Visualization.renderVisualizationsTemplate,
       replace: false,
       link: function (scope) {
-
         scope.select2Options = {
           width: '50%'
         };
@@ -103,7 +102,7 @@
           if (filters) {
             scope.filters = filters;
 
-            if (filters.length) {
+            if (filters.length && false) {
               readURL();
 
               // start watching for changes and update URL

rd_ui/app/scripts/visualizations/chart.js

@@ -33,7 +33,7 @@
         $scope.chartSeries = [];
         $scope.chartOptions = {};
 
-        var reloadData = _.throttle(function(data) {
+        var reloadData = function(data) {
           if (!data || ($scope.queryResult && $scope.queryResult.getData()) == null) {
             $scope.chartSeries.splice(0, $scope.chartSeries.length);
           } else {
@@ -49,8 +49,8 @@
               }
               $scope.chartSeries.push(_.extend(s, additional));
             });
-          }
-        }, 500);
+          };
+        };
 
         $scope.$watch('options', function (chartOptions) {
           if (chartOptions) {
@@ -87,6 +87,8 @@
           'Pie': 'pie'
         };
 
+        scope.globalSeriesType = 'column';
+
         scope.stackingOptions = {
           "None": "none",
           "Normal": "normal",
@@ -120,6 +122,13 @@
         var chartOptionsUnwatch = null,
             columnsWatch = null;
 
+        scope.$watch('globalSeriesType', function(type, old) {
+          if (type && old && type !== old && scope.visualization.options.seriesOptions) {
+            _.each(scope.visualization.options.seriesOptions, function(sOptions) {
+              sOptions.type = type;
+            });
+          }
+        });
         scope.$watch('visualization.type', function (visualizationType) {
           if (visualizationType == 'CHART') {
             if (scope.visualization.options.series.stacking === null) {
@@ -135,7 +144,9 @@
 
               // TODO: remove uneeded ones?
               if (scope.visualization.options.seriesOptions == undefined) {
-                scope.visualization.options.seriesOptions = {};
+                scope.visualization.options.seriesOptions = {
+                  type: scope.globalSeriesType
+                };
               };
 
               _.each(scope.series, function(s, i) {
@@ -230,4 +241,4 @@
       }
     }
   });
-}());
+}());

rd_ui/app/styles/login.css

@@ -1,14 +1,15 @@
 .main {
     max-width: 320px;
     margin: 0 auto;
+    margin-top:20px;
 }
 
 .login-or {
     position: relative;
     font-size: 18px;
     color: #aaa;
-    margin-top: 10px;
-            margin-bottom: 10px;
+    margin-top: 20px;
+    margin-bottom: 20px;
     padding-top: 10px;
     padding-bottom: 10px;
 }
@@ -31,7 +32,9 @@
     margin-bottom: 0px !important;
 }
 
-/*h3 {*/
-    /*text-align: center;*/
-    /*line-height: 300%;*/
-/*}*/
+img.login-button {
+    width: 250px;
+    display: block;
+    margin-left: auto;
+    margin-right: auto;
+}

rd_ui/app/views/dashboard.html

@@ -29,7 +29,7 @@
                             <span ng-hide="currentUser.hasPermission('view_query')">{{query.name}}</span>
                             <query-link query="query" visualization="widget.visualization" ng-show="currentUser.hasPermission('view_query')"></query-link>
                         </p>
-                        <div class="text-muted" ng-bind="query.description"></div>
+                        <div class="text-muted" ng-bind-html="query.description | markdown"></div>
                     </h3>
                 </div>
 

rd_ui/app/views/queries_search_results.html

@@ -0,0 +1,19 @@
+<div class="container">
+    <div class="row">
+        <p>
+            <form class="form-inline" role="form" ng-submit="search()">
+                <div class="form-group">
+                    <input class="form-control" placeholder="Search..." ng-model="term" autofocus>
+                </div>
+                <button type="submit" class="btn btn-primary">
+                    <span class="glyphicon glyphicon-search"></span>
+                </button>
+            </form>
+        </p>
+
+        <smart-table rows="queries" columns="gridColumns"
+                 config="gridConfig"
+                 class="table table-condensed table-hover"></smart-table>
+    </div>
+
+</div>

rd_ui/app/views/query.html

@@ -12,7 +12,14 @@
                     </h2>
                     <p>
                         <em>
-                            <edit-in-place editable="isQueryOwner" done="saveDescription" editor="textarea" placeholder="No description" ignore-blanks='false' value="query.description"></edit-in-place>
+                            <edit-in-place editable="isQueryOwner"
+                                           done="saveDescription"
+                                           editor="textarea"
+                                           placeholder="No description"
+                                           ignore-blanks='false'
+                                           value="query.description"
+                                           markdown="true">
+                             </edit-in-place>
                         </em>
                     </p>
                 </div>

rd_ui/app/views/visualizations/chart_editor.html

@@ -18,6 +18,15 @@
                             class="form-control"></select>
                 </div>
             </div>
+
+            <div class="form-group">
+                <label class="control-label col-sm-2">Series Type</label>
+
+                <div class="col-sm-10">
+                    <select required ng-options="value as key for (key, value) in seriesTypes"
+                            ng-model="globalSeriesType" class="form-control"></select>
+                </div>
+            </div>
         </div>
     </div>
 
@@ -88,4 +97,4 @@
 
         </div>
     </div>
-</div>
+</div>

rd_ui/app/views/visualizations/edit_visualization.html

@@ -24,4 +24,4 @@
     </div>
 
   </form>
-</div>
+</div>

rd_ui/test/unit/test_visualization_renderer.js

@@ -54,7 +54,7 @@ describe('VisualizationRenderer', function() {
   });
 
 
-  describe('URL binding', function() {
+  /*describe('URL binding', function() {
 
     beforeEach(inject(function($rootScope, $compile, $location) {
       spyOn($location, 'search').andCallThrough();
@@ -85,5 +85,5 @@ describe('VisualizationRenderer', function() {
         var searchFilters = angular.fromJson($location.search().filters);
         expect(searchFilters[filters[0].friendlyName]).toEqual('newValue');
     }));
-  });
+  });*/
 });

redash/__init__.py

@@ -3,7 +3,7 @@ import urlparse
 import redis
 from statsd import StatsClient
 
-from redash import settings, events
+from redash import settings
 
 __version__ = '0.4.0'
 
@@ -15,8 +15,6 @@ def setup_logging():
     logging.getLogger().addHandler(handler)
     logging.getLogger().setLevel(settings.LOG_LEVEL)
 
-    events.setup_logging(settings.EVENTS_LOG_PATH, settings.EVENTS_CONSOLE_OUTPUT)
-
 
 def create_redis_connection():
     redis_url = urlparse.urlparse(settings.REDIS_URL)

redash/authentication.py

@@ -6,10 +6,8 @@ import logging
 
 from flask import request, make_response, redirect, url_for
 from flask.ext.login import LoginManager, login_user, current_user
-from flask.ext.googleauth import GoogleAuth, login
-from werkzeug.contrib.fixers import ProxyFix
 
-from redash import models, settings
+from redash import models, settings, google_oauth
 
 login_manager = LoginManager()
 logger = logging.getLogger('authentication')
@@ -57,48 +55,15 @@ class HMACAuthentication(object):
         return decorated
 
 
-def validate_email(email):
-    if not settings.GOOGLE_APPS_DOMAIN:
-        return True
-
-    return email in settings.ALLOWED_EXTERNAL_USERS or email.endswith("@%s" % settings.GOOGLE_APPS_DOMAIN)
-
-
-def create_and_login_user(app, user):
-    if not validate_email(user.email):
-        return
-
-    try:
-        user_object = models.User.get(models.User.email == user.email)
-        if user_object.name != user.name:
-            logger.debug("Updating user name (%r -> %r)", user_object.name, user.name)
-            user_object.name = user.name
-            user_object.save()
-    except models.User.DoesNotExist:
-        logger.debug("Creating user object (%r)", user.name)
-        user_object = models.User.create(name=user.name, email=user.email, groups = models.User.DEFAULT_GROUPS)
-
-    login_user(user_object, remember=True)
-
-login.connect(create_and_login_user)
-
-
 @login_manager.user_loader
 def load_user(user_id):
     return models.User.select().where(models.User.id == user_id).first()
 
 
 def setup_authentication(app):
-    if settings.GOOGLE_OPENID_ENABLED:
-        openid_auth = GoogleAuth(app, url_prefix="/google_auth")
-        # If we don't have a list of external users, we can use Google's federated login, which limits
-        # the domain with which you can sign in.
-        if not settings.ALLOWED_EXTERNAL_USERS and settings.GOOGLE_APPS_DOMAIN:
-            openid_auth._OPENID_ENDPOINT = "https://www.google.com/a/%s/o8/ud?be=o8" % settings.GOOGLE_APPS_DOMAIN
-
     login_manager.init_app(app)
     login_manager.anonymous_user = models.AnonymousUser
-    app.wsgi_app = ProxyFix(app.wsgi_app)
     app.secret_key = settings.COOKIE_SECRET
+    app.register_blueprint(google_oauth.blueprint)
 
     return HMACAuthentication()

redash/controllers.py

@@ -10,23 +10,20 @@ import json
 import numbers
 import cStringIO
 import datetime
+import logging
 
 from flask import render_template, send_from_directory, make_response, request, jsonify, redirect, \
     session, url_for
 from flask.ext.restful import Resource, abort
 from flask_login import current_user, login_user, logout_user
-
 import sqlparse
-import events
-from permissions import require_permission
 
 from redash import redis_connection, statsd_client, models, settings, utils, __version__
 from redash.wsgi import app, auth, api
+from redash.tasks import QueryTask, record_event
+from redash.cache import headers as cache_headers
+from redash.permissions import require_permission
 
-import logging
-from tasks import QueryTask
-
-from cache import headers as cache_headers
 
 @app.route('/ping', methods=['GET'])
 def ping():
@@ -69,8 +66,7 @@ def login():
         return redirect(request.args.get('next') or '/')
 
     if not settings.PASSWORD_LOGIN_ENABLED:
-        blueprint = app.extensions['googleauth'].blueprint
-        return redirect(url_for("%s.login" % blueprint.name, next=request.args.get('next')))
+        return redirect(url_for("google_oauth.authorize", next=request.args.get('next')))
 
     if request.method == 'POST':
         user = models.User.select().where(models.User.email == request.form['username']).first()
@@ -84,7 +80,7 @@ def login():
                            analytics=settings.ANALYTICS,
                            next=request.args.get('next'),
                            username=request.form.get('username', ''),
-                           show_google_openid=settings.GOOGLE_OPENID_ENABLED)
+                           show_google_openid=settings.GOOGLE_OAUTH_ENABLED)
 
 
 @app.route('/logout')
@@ -111,7 +107,6 @@ def status_api():
 
     manager_status = redis_connection.hgetall('redash:status')
     status['manager'] = manager_status
-    status['manager']['queue_size'] = redis_connection.llen('queries') + redis_connection.llen('scheduled_queries')
     status['manager']['outdated_queries_count'] = models.Query.outdated_queries().count()
 
     queues = {}
@@ -160,7 +155,7 @@ class EventAPI(BaseResource):
     def post(self):
         events_list = request.get_json(force=True)
         for event in events_list:
-            events.record_event(event)
+            record_event.delay(event)
 
 
 api.add_resource(EventAPI, '/api/events', endpoint='events')
@@ -281,6 +276,14 @@ api.add_resource(WidgetListAPI, '/api/widgets', endpoint='widgets')
 api.add_resource(WidgetAPI, '/api/widgets/<int:widget_id>', endpoint='widget')
 
 
+class QuerySearchAPI(BaseResource):
+    @require_permission('view_query')
+    def get(self):
+        term = request.args.get('q', '')
+
+        return [q.to_dict() for q in models.Query.search(term)]
+
+
 class QueryListAPI(BaseResource):
     @require_permission('create_query')
     def post(self):
@@ -329,6 +332,7 @@ class QueryAPI(BaseResource):
         else:
             abort(404, message="Query not found.")
 
+api.add_resource(QuerySearchAPI, '/api/queries/search', endpoint='queries_search')
 api.add_resource(QueryListAPI, '/api/queries', endpoint='queries')
 api.add_resource(QueryAPI, '/api/queries/<query_id>', endpoint='query')
 
@@ -414,48 +418,52 @@ class QueryResultListAPI(BaseResource):
 
 
 class QueryResultAPI(BaseResource):
-    @require_permission('view_query')
-    def get(self, query_result_id):
-        query_result = models.QueryResult.get_by_id(query_result_id)
-        if query_result:
-            data = json.dumps({'query_result': query_result.to_dict()}, cls=utils.JSONEncoder)
-            return make_response(data, 200, cache_headers)
-        else:
-            abort(404)
+    @staticmethod
+    def csv_response(query_result):
+        s = cStringIO.StringIO()
 
+        query_data = json.loads(query_result.data)
+        writer = csv.DictWriter(s, fieldnames=[col['name'] for col in query_data['columns']])
+        writer.writer = utils.UnicodeWriter(s)
+        writer.writeheader()
+        for row in query_data['rows']:
+            for k, v in row.iteritems():
+                if isinstance(v, numbers.Number) and (v > 1000 * 1000 * 1000 * 100):
+                    row[k] = datetime.datetime.fromtimestamp(v/1000.0)
+
+            writer.writerow(row)
+
+        headers = {'Content-Type': "text/csv; charset=UTF-8"}
+        headers.update(cache_headers)
+        return make_response(s.getvalue(), 200, headers)
 
-class CsvQueryResultsAPI(BaseResource):
     @require_permission('view_query')
-    def get(self, query_id, query_result_id=None):
-        if not query_result_id:
+    def get(self, query_id=None, query_result_id=None, filetype='json'):
+        if query_result_id is None and query_id is not None:
             query = models.Query.get(models.Query.id == query_id)
             if query:
                 query_result_id = query._data['latest_query_data']
 
-        query_result = query_result_id and models.QueryResult.get_by_id(query_result_id)
+        if query_result_id:
+            query_result = models.QueryResult.get_by_id(query_result_id)
+
         if query_result:
-            s = cStringIO.StringIO()
+            if filetype == 'json':
+                data = json.dumps({'query_result': query_result.to_dict()}, cls=utils.JSONEncoder)
+                return make_response(data, 200, cache_headers)
+            else:
+                return self.csv_response(query_result)
 
-            query_data = json.loads(query_result.data)
-            writer = csv.DictWriter(s, fieldnames=[col['name'] for col in query_data['columns']])
-            writer.writer = utils.UnicodeWriter(s)
-            writer.writeheader()
-            for row in query_data['rows']:
-                for k, v in row.iteritems():
-                    if isinstance(v, numbers.Number) and (v > 1000 * 1000 * 1000 * 100):
-                        row[k] = datetime.datetime.fromtimestamp(v/1000.0)
-
-                writer.writerow(row)
-
-            return make_response(s.getvalue(), 200, {'Content-Type': "text/csv; charset=UTF-8"})
         else:
             abort(404)
 
-api.add_resource(CsvQueryResultsAPI, '/api/queries/<query_id>/results/<query_result_id>.csv',
-                 '/api/queries/<query_id>/results.csv',
-                 endpoint='csv_query_results')
+
 api.add_resource(QueryResultListAPI, '/api/query_results', endpoint='query_results')
-api.add_resource(QueryResultAPI, '/api/query_results/<query_result_id>', endpoint='query_result')
+api.add_resource(QueryResultAPI,
+                 '/api/query_results/<query_result_id>',
+                 '/api/queries/<query_id>/results.<filetype>',
+                 '/api/queries/<query_id>/results/<query_result_id>.<filetype>',
+                 endpoint='query_result')
 
 
 class JobAPI(BaseResource):

redash/data/query_runner_mysql.py

@@ -18,7 +18,7 @@ def mysql(connection_string):
     
     def query_runner(query):
         connections_params = [entry.split('=')[1] for entry in connection_string.split(';')]
-        connection = MySQLdb.connect(*connections_params)
+        connection = MySQLdb.connect(*connections_params, charset="utf8", use_unicode=True)
         cursor = connection.cursor()
 
         logging.debug("mysql got query: %s", query)
@@ -61,4 +61,4 @@ def mysql(connection_string):
         return json_data, error
         
     
-    return query_runner
+    return query_runner

redash/events.py

@@ -1,23 +0,0 @@
-import logging
-import json
-
-logger = logging.getLogger("redash.events")
-logger.propagate = False
-
-
-def setup_logging(log_path, console_output=False):
-    if log_path:
-        fh = logging.FileHandler(log_path)
-        formatter = logging.Formatter('%(message)s')
-        fh.setFormatter(formatter)
-        logger.addHandler(fh)
-
-    if console_output:
-        handler = logging.StreamHandler()
-        formatter = logging.Formatter('[%(name)s] %(message)s')
-        handler.setFormatter(formatter)
-        logger.addHandler(handler)
-
-
-def record_event(event):
-    logger.info(json.dumps(event))

redash/google_oauth.py

@@ -0,0 +1,81 @@
+import logging
+from flask.ext.login import login_user
+import requests
+from flask import redirect, url_for, Blueprint
+from flask_oauth import OAuth
+from redash import models, settings
+
+logger = logging.getLogger('google_oauth')
+oauth = OAuth()
+
+request_token_params = {'scope': 'https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/userinfo.profile', 'response_type': 'code'}
+
+if settings.GOOGLE_APPS_DOMAIN:
+    request_token_params['hd'] = settings.GOOGLE_APPS_DOMAIN
+else:
+    logger.warning("No Google Apps domain defined, all Google accounts allowed.")
+
+google = oauth.remote_app('google',
+                          base_url='https://www.google.com/accounts/',
+                          authorize_url='https://accounts.google.com/o/oauth2/auth',
+                          request_token_url=None,
+                          request_token_params=request_token_params,
+                          access_token_url='https://accounts.google.com/o/oauth2/token',
+                          access_token_method='POST',
+                          access_token_params={'grant_type': 'authorization_code'},
+                          consumer_key=settings.GOOGLE_CLIENT_ID,
+                          consumer_secret=settings.GOOGLE_CLIENT_SECRET)
+
+
+blueprint = Blueprint('google_oauth', __name__)
+
+
+def get_user_profile(access_token):
+    headers = {'Authorization': 'OAuth '+access_token}
+    response = requests.get('https://www.googleapis.com/oauth2/v1/userinfo', headers=headers)
+
+    if response.status_code == 401:
+        logger.warning("Failed getting user profile (response code 401).")
+        return None
+
+    return response.json()
+
+
+def create_and_login_user(name, email):
+    try:
+        user_object = models.User.get(models.User.email == email)
+        if user_object.name != name:
+            logger.debug("Updating user name (%r -> %r)", user_object.name, name)
+            user_object.name = name
+            user_object.save()
+    except models.User.DoesNotExist:
+        logger.debug("Creating user object (%r)", name)
+        user_object = models.User.create(name=name, email=email, groups=models.User.DEFAULT_GROUPS)
+
+    login_user(user_object, remember=True)
+
+
+@blueprint.route('/oauth/google', endpoint="authorize")
+def login():
+    # TODO, suport next
+    callback=url_for('.callback', _external=True)
+    logger.debug("Callback url: %s", callback)
+    return google.authorize(callback=callback)
+
+
+@blueprint.route('/oauth/google_callback', endpoint="callback")
+@google.authorized_handler
+def authorized(resp):
+    access_token = resp['access_token']
+
+    if access_token is None:
+        logger.warning("Access token missing in call back request.")
+        return redirect(url_for('login'))
+
+    profile = get_user_profile(access_token)
+    if profile is None:
+        return redirect(url_for('login'))
+
+    create_and_login_user(profile['name'], profile['email'])
+
+    return redirect(url_for('index'))

redash/models.py

@@ -348,6 +348,11 @@ class Query(BaseModel):
 
         return queries
 
+    @classmethod
+    def search(cls, term):
+        # This is very naive implementation of search, to be replaced with PostgreSQL full-text-search solution.
+        return cls.select().where((cls.name**"%{}%".format(term)) | (cls.description**"%{}%".format(term)))
+
     @classmethod
     def update_instance(cls, query_id, **kwargs):
         if 'query' in kwargs:
@@ -520,6 +525,21 @@ class Event(BaseModel):
     def __unicode__(self):
         return u"%s,%s,%s,%s" % (self._data['user'], self.action, self.object_type, self.object_id)
 
+    @classmethod
+    def record(cls, event):
+        user = event.pop('user_id')
+        action = event.pop('action')
+        object_type = event.pop('object_type')
+        object_id = event.pop('object_id', None)
+
+        created_at = datetime.datetime.utcfromtimestamp(event.pop('timestamp'))
+        additional_properties = json.dumps(event)
+
+        event = cls.create(user=user, action=action, object_type=object_type, object_id=object_id,
+                           additional_properties=additional_properties, created_at=created_at)
+
+        return event
+
 
 all_models = (DataSource, User, QueryResult, Query, Dashboard, Visualization, Widget, ActivityLog, Group, Event)
 

redash/settings.py

@@ -59,18 +59,19 @@ CELERY_FLOWER_URL = os.environ.get("REDASH_CELERY_FLOWER_URL", "/flower")
 # Google Apps domain to allow access from; any user with email in this Google Apps will be allowed
 # access
 GOOGLE_APPS_DOMAIN = os.environ.get("REDASH_GOOGLE_APPS_DOMAIN", "")
-GOOGLE_OPENID_ENABLED = parse_boolean(os.environ.get("REDASH_GOOGLE_OPENID_ENABLED", "true"))
-PASSWORD_LOGIN_ENABLED = parse_boolean(os.environ.get("REDASH_PASSWORD_LOGIN_ENABLED", "false"))
-ALLOWED_EXTERNAL_USERS = array_from_string(os.environ.get("REDASH_ALLOWED_EXTERNAL_USERS", ''))
+
+GOOGLE_CLIENT_ID = os.environ.get("REDASH_GOOGLE_CLIENT_ID", "")
+GOOGLE_CLIENT_SECRET = os.environ.get("REDASH_GOOGLE_CLIENT_SECRET", "")
+GOOGLE_OAUTH_ENABLED = GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET
+
+PASSWORD_LOGIN_ENABLED = parse_boolean(os.environ.get("REDASH_PASSWORD_LOGIN_ENABLED", "true"))
 STATIC_ASSETS_PATH = fix_assets_path(os.environ.get("REDASH_STATIC_ASSETS_PATH", "../rd_ui/app/"))
 WORKERS_COUNT = int(os.environ.get("REDASH_WORKERS_COUNT", "2"))
-JOB_EXPIRY_TIME = int(os.environ.get("REDASH_JOB_EXPIRY_TIME", 3600*24))
+JOB_EXPIRY_TIME = int(os.environ.get("REDASH_JOB_EXPIRY_TIME", 3600*6))
 COOKIE_SECRET = os.environ.get("REDASH_COOKIE_SECRET", "c292a0a3aa32397cdb050e233733900f")
 LOG_LEVEL = os.environ.get("REDASH_LOG_LEVEL", "INFO")
-EVENTS_LOG_PATH = os.environ.get("REDASH_EVENTS_LOG_PATH", "")
-EVENTS_CONSOLE_OUTPUT = parse_boolean(os.environ.get("REDASH_EVENTS_CONSOLE_OUTPUT", "false"))
 CLIENT_SIDE_METRICS = parse_boolean(os.environ.get("REDASH_CLIENT_SIDE_METRICS", "false"))
 ANALYTICS = os.environ.get("REDASH_ANALYTICS", "")
 
 # Features:
-FEATURE_TABLES_PERMISSIONS = parse_boolean(os.environ.get("REDASH_FEATURE_TABLES_PERMISSIONS", "false"))
+FEATURE_TABLES_PERMISSIONS = parse_boolean(os.environ.get("REDASH_FEATURE_TABLES_PERMISSIONS", "false"))

redash/tasks.py

@@ -5,7 +5,7 @@ import redis
 from celery import Task
 from celery.result import AsyncResult
 from celery.utils.log import get_task_logger
-from redash import redis_connection, models, statsd_client
+from redash import redis_connection, models, statsd_client, settings
 from redash.utils import gen_query_hash
 from redash.worker import celery
 from redash.data.query_runner import get_query_runner
@@ -64,7 +64,12 @@ class QueryTask(object):
                     logging.info("[Manager][%s] Found existing job: %s", query_hash, job_id)
 
                     job = cls(job_id=job_id)
-                else:
+                    if job.ready():
+                        logging.info("[%s] job found is ready (%s), removing lock", query_hash, job.celery_status)
+                        redis_connection.delete(QueryTask._job_lock_id(query_hash, data_source.id))
+                        job = None
+
+                if not job:
                     pipe.multi()
 
                     if scheduled:
@@ -75,7 +80,7 @@ class QueryTask(object):
                     result = execute_query.apply_async(args=(query, data_source.id), queue=queue_name)
                     job = cls(async_result=result)
                     logging.info("[Manager][%s] Created new job: %s", query_hash, job.id)
-                    pipe.set(cls._job_lock_id(query_hash, data_source.id), job.id)
+                    pipe.set(cls._job_lock_id(query_hash, data_source.id), job.id, settings.JOB_EXPIRY_TIME)
                     pipe.execute()
                 break
 
@@ -113,6 +118,17 @@ class QueryTask(object):
             'query_result_id': query_result_id,
         }
 
+    @property
+    def is_cancelled(self):
+        return self._async_result.status == 'REVOKED'
+
+    @property
+    def celery_status(self):
+        return self._async_result.status
+
+    def ready(self):
+        return self._async_result.ready()
+
     def cancel(self):
         return self._async_result.revoke(terminate=True)
 
@@ -151,6 +167,41 @@ def refresh_queries():
     statsd_client.gauge('manager.seconds_since_refresh', now - float(status.get('last_refresh_at', now)))
 
 
+@celery.task(base=BaseTask)
+def cleanup_tasks():
+    # in case of cold restart of the workers, there might be jobs that still have their "lock" object, but aren't really
+    # going to run. this job removes them.
+
+    lock_keys = redis_connection.keys("query_hash_job:*") # TODO: use set instead of keys command
+    query_tasks = [QueryTask(job_id=j) for j in redis_connection.mget(lock_keys)]
+
+    logger.info("Found %d locks", len(query_tasks))
+
+    inspect = celery.control.inspect()
+    active_tasks = inspect.active()
+    if active_tasks is None:
+        active_tasks = []
+    else:
+        active_tasks = active_tasks.values()
+
+    all_tasks = set()
+    for task_list in active_tasks:
+        for task in task_list:
+            all_tasks.add(task['id'])
+
+    logger.info("Active jobs count: %d", len(all_tasks))
+
+    for i, t in enumerate(query_tasks):
+        if t.ready():
+            # if locked task is ready already (failed, finished, revoked), we don't need the lock anymore
+            logger.warning("%s is ready (%s), removing lock.", lock_keys[i], t.celery_status)
+            redis_connection.delete(lock_keys[i])
+
+        if t.celery_status == 'STARTED' and t.id not in all_tasks:
+            logger.warning("Couldn't find active job for: %s, removing lock.", lock_keys[i])
+            redis_connection.delete(lock_keys[i])
+
+
 @celery.task(bind=True, base=BaseTask, track_started=True)
 def execute_query(self, query, data_source_id):
     # TODO: maybe this should be a class?
@@ -195,3 +246,7 @@ def execute_query(self, query, data_source_id):
 
     return query_result.id
 
+
+@celery.task(base=BaseTask)
+def record_event(event):
+    models.Event.record(event)

redash/worker.py

@@ -13,6 +13,10 @@ celery.conf.update(CELERY_RESULT_BACKEND=settings.CELERY_BACKEND,
                            'task': 'redash.tasks.refresh_queries',
                            'schedule': timedelta(seconds=30)
                        },
+                       'cleanup_tasks': {
+                           'task': 'redash.tasks.cleanup_tasks',
+                           'schedule': timedelta(minutes=5)
+                       },
                    },
                    CELERY_TIMEZONE='UTC')
 

requirements.txt

@@ -1,7 +1,7 @@
 Flask==0.10.1
-Flask-GoogleAuth==0.4
 Flask-RESTful==0.2.10
 Flask-Login==0.2.9
+Flask-OAuth==0.12
 passlib==1.6.2
 Jinja2==2.7.2
 MarkupSafe==0.18
@@ -10,7 +10,7 @@ aniso8601==0.82
 blinker==1.3
 itsdangerous==0.23
 peewee==2.2.2
-psycopg2==2.5.1
+psycopg2==2.5.2
 python-dateutil==2.1
 pytz==2013.9
 redis==2.7.5

tests/test_authentication.py

@@ -1,52 +1,25 @@
-from unittest import TestCase
 from mock import patch
-from flask_googleauth import ObjectDict
 from tests import BaseTestCase
-from redash.authentication import validate_email, create_and_login_user
-from redash import settings, models
+from redash import models
+from redash.google_oauth import create_and_login_user
 from tests.factories import user_factory
 
 
-class TestEmailValidation(TestCase):
-    def test_accepts_address_with_correct_domain(self):
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', 'example.com'):
-            self.assertTrue(validate_email('example@example.com'))
-
-    def test_accepts_address_from_exception_list(self):
-        with patch.multiple(settings, GOOGLE_APPS_DOMAIN='example.com', ALLOWED_EXTERNAL_USERS=['whatever@whatever.com']):
-            self.assertTrue(validate_email('whatever@whatever.com'))
-
-    def test_accept_any_address_when_domain_empty(self):
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', None):
-            self.assertTrue(validate_email('whatever@whatever.com'))
-
-    def test_rejects_address_with_incorrect_domain(self):
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', 'example.com'):
-            self.assertFalse(validate_email('whatever@whatever.com'))
-
-
 class TestCreateAndLoginUser(BaseTestCase):
     def test_logins_valid_user(self):
         user = user_factory.create(email='test@example.com')
 
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', 'example.com'), patch('redash.authentication.login_user') as login_user_mock:
-            create_and_login_user(None, user)
+        with patch('redash.google_oauth.login_user') as login_user_mock:
+            create_and_login_user(user.name, user.email)
             login_user_mock.assert_called_once_with(user, remember=True)
 
     def test_creates_vaild_new_user(self):
-        openid_user = ObjectDict({'email': 'test@example.com', 'name': 'Test User'})
+        email = 'test@example.com'
+        name = 'Test User'
 
-        with patch.multiple(settings, GOOGLE_APPS_DOMAIN='example.com'), \
-             patch('redash.authentication.login_user') as login_user_mock:
+        with patch('redash.google_oauth.login_user') as login_user_mock:
 
-            create_and_login_user(None, openid_user)
+            create_and_login_user(name, email)
 
             self.assertTrue(login_user_mock.called)
-            user = models.User.get(models.User.email == openid_user.email)
-
-    def test_ignores_invliad_user(self):
-        user = ObjectDict({'email': 'test@whatever.com'})
-
-        with patch.object(settings, 'GOOGLE_APPS_DOMAIN', 'example.com'), patch('redash.authentication.login_user') as login_user_mock:
-            create_and_login_user(None, user)
-            self.assertFalse(login_user_mock.called)
+            user = models.User.get(models.User.email == email)

tests/test_controllers.py

@@ -380,7 +380,7 @@ class TestLogin(BaseTestCase):
         with app.test_client() as c, patch.object(settings, 'PASSWORD_LOGIN_ENABLED', False):
             rv = c.get('/login')
             self.assertEquals(rv.status_code, 302)
-            self.assertTrue(rv.location.endswith(url_for('GoogleAuth.login')))
+            self.assertTrue(rv.location.endswith(url_for('google_oauth.authorize')))
 
     def test_get_login_form(self):
         with app.test_client() as c:

tests/test_models.py

@@ -1,7 +1,8 @@
 import datetime
+import json
 from tests import BaseTestCase
 from redash import models
-from factories import dashboard_factory, query_factory, data_source_factory, query_result_factory
+from factories import dashboard_factory, query_factory, data_source_factory, query_result_factory, user_factory
 from redash.utils import gen_query_hash
 
 
@@ -29,6 +30,28 @@ class QueryTest(BaseTestCase):
 
         self.assertNotEquals(old_hash, q.query_hash)
 
+    def test_search_finds_in_name(self):
+        q1 = query_factory.create(name="Testing search")
+        q2 = query_factory.create(name="Testing searching")
+        q3 = query_factory.create(name="Testing sea rch")
+
+        queries = models.Query.search("search")
+
+        self.assertIn(q1, queries)
+        self.assertIn(q2, queries)
+        self.assertNotIn(q3, queries)
+
+    def test_search_finds_in_description(self):
+        q1 = query_factory.create(description="Testing search")
+        q2 = query_factory.create(description="Testing searching")
+        q3 = query_factory.create(description="Testing sea rch")
+
+        queries = models.Query.search("search")
+
+        self.assertIn(q1, queries)
+        self.assertIn(q2, queries)
+        self.assertNotIn(q3, queries)
+
 
 class QueryResultTest(BaseTestCase):
     def setUp(self):
@@ -93,6 +116,7 @@ class QueryResultTest(BaseTestCase):
 
         self.assertEqual(found_query_result.id, qr.id)
 
+
 class TestQueryResultStoreResult(BaseTestCase):
     def setUp(self):
         super(TestQueryResultStoreResult, self).setUp()
@@ -148,4 +172,38 @@ class TestQueryResultStoreResult(BaseTestCase):
 
         self.assertEqual(models.Query.get_by_id(query1.id)._data['latest_query_data'], query_result.id)
         self.assertEqual(models.Query.get_by_id(query2.id)._data['latest_query_data'], query_result.id)
-        self.assertNotEqual(models.Query.get_by_id(query3.id)._data['latest_query_data'], query_result.id)
+        self.assertNotEqual(models.Query.get_by_id(query3.id)._data['latest_query_data'], query_result.id)
+
+
+class TestEvents(BaseTestCase):
+    def raw_event(self):
+        timestamp = 1411778709.791
+        user = user_factory.create()
+        created_at = datetime.datetime.utcfromtimestamp(timestamp)
+        raw_event = {"action": "view",
+                      "timestamp": timestamp,
+                      "object_type": "dashboard",
+                      "user_id": user.id,
+                      "object_id": 1}
+
+        return raw_event, user, created_at
+
+    def test_records_event(self):
+        raw_event, user, created_at = self.raw_event()
+
+        event = models.Event.record(raw_event)
+
+        self.assertEqual(event.user, user)
+        self.assertEqual(event.action, "view")
+        self.assertEqual(event.object_type, "dashboard")
+        self.assertEqual(event.object_id, 1)
+        self.assertEqual(event.created_at, created_at)
+
+    def test_records_additional_properties(self):
+        raw_event, _, _ = self.raw_event()
+        additional_properties = {'test': 1, 'test2': 2, 'whatever': "abc"}
+        raw_event.update(additional_properties)
+
+        event = models.Event.record(raw_event)
+
+        self.assertDictEqual(json.loads(event.additional_properties), additional_properties)