Home > node.js > Deep database schema for a link shortener with statistics

Deep database schema for a link shortener with statistics

January 11Hits:0
Advertisement

I want to create a link shortener and I'm still trying to figure out how I could track every single view. I'm using NodeJS, MongoDB and mongoose. The main reason that my following schemas are so deeply nested is that I want to have a system which can show data about a specific link, the whole user and in both cases about specific days. All time stats can be instantly accessed without iterating over thousands of database entries too.

Everyone who wants to share links has to create an account with the following schema:

{     // User.js     // some other unimportant properties      // all time stats for the user     stats: {         views           : {             type   : Number,             default: 0         },         uniqueViews      : {             type   : Number,             default: 0         }     },      days: { // This is explained below         type   : mongoose.Schema.Types.Mixed,         default: {}     } } 

The link schema looks like this:

{     // Link.js     // some other unimportant properties     user: {         type: String,         ref : 'User'     },      stats: {         views     : {             type   : Number,             default: 0         },         uniqueViews: {             type   : Number,             default: 0         }     },      days: { // This is explained below         type   : mongoose.Schema.Types.Mixed,         default: {}     } } 


Every time when a user visits a link generated by user, I want to track the impression. To see statistics about every day or for example the last 7 days, I have a (dynamic) days field in Link.js and User.js.

The day schemas would look similar to this if you would have taken a snapshot from the database:

// User.js days: {     '1456095600000': { // new Date() with cleared time and getTime()         views     : 20,         validViews: 18,          device: {             'desktop': {                 views     : 20,                 validViews: 18,             },             'tablet': {                 views     : 14,                 validViews: 12,             },             'mobile': {                 views     : 45,                 validViews: 40,             }         },          country: {             // etc.         }     }     // a bunch of other days }   // Link.js days: {     '1456095600000': {         views     : 3,         validViews: 2,         earnings  : 0.0045,          device: {             'desktop': {                 views     : 20,                 validViews: 18,             },             'tablet': {                 views     : 14,                 validViews: 12,             },             'mobile': {                 views     : 45,                 validViews: 40,             }         },          country: {             // etc.         }     }     // a bunch of other days } 


Now finally the route where the actual tracking is happening:

app.get('/:id', function (req, res, next) {      // redirect the user first, then track     res.redirect(link.destination);      var today = new Date();     today.setHours(0,0,0,0);     var day = today.getTime(); // now we have "today 0:00" in ms      // some (magic) tracking data, like country or device     var device = req.device, country = req.country;      // checking for a (magic) "valid" view (a valid view is one view per session)     var isValid = !!req.valid;      Link.findById(req.params.id).populate('user').exec(function (err, link) {         // user first:         link.user.stats.views++;         if (isValid) link.user.stats.uniqueViews++;         link.save();          // assume that the following objects were initialized before.          link.user.days[day].views++;         if (isValid) link.user.days[day].validViews++;          link.user.days[day].device[device].views++;         if (isValid) link.user.days[day].device[device].validViews++;          // and so on with `country` and `referrer`          // the same now for the link:          link.stats.views++;         if (isValid) link.stats.uniqueViews++;          // assume that the following objects were initialized before.          link.days[day].views++;         if (isValid) link.user.days[day].validViews++;          link.days[day].device[device].views++;         if (isValid) link.days[day].device[device].validViews++;          // and so on with `country` and `referrer`          // mark the mixed objects as modified and save          link.markModified('days');         link.markModified('user.days');         link.save();         link.user.save();     }); }); 

Am I doing it right or would you do it completely different? A friend of mine asked whether MongoDB is even the right database for that, but I think this is not a problem, is it? When doing it like I described, the variables with dozens of links are eventually really big. Can this be a problem (assuming I have 6 powerful cores (with the app clustered of course), 40GB RAM and a SSD)?

Related Articles

  • Deep database schema for a link shortener with statisticsJanuary 11

    I want to create a link shortener and I'm still trying to figure out how I could track every single view. I'm using NodeJS, MongoDB and mongoose. The main reason that my following schemas are so deeply nested is that I want to have a system which can

  • NodeJS link shortenerFebruary 7

    I made an app in NodeJS, with Express and Mongoose, that takes a URL as input into the address bar, and returns an alias to that URL. It can be used as a link shortener. The app is intended to fulfill this challenge's user stories: I can pass a URL a

  • How do you update your production codebase/database schema without causing downtime?January 9

    What are some techniques for updating a production server's code base / database schema without causing any downtime? --------------Solutions------------- For typical databases (Oracle for example) it's possible to alter the database schema while sti

  • MySQL Database Schema for Train Timetables

    MySQL Database Schema for Train TimetablesFebruary 22

    I am interested in what a MySQL database schema would look like for a train timetable. Generally this is presented in table format as the final result. Train No. 11111 22222 11111 Train Day Mo-Fr Sat Sun Station A d 06.00 07.00 07.00 Station B d 06.1

  • database schema with unlimited hierarchical data that is modified a lotMay 11

    I am trying to set up some sort of collaboration app/forum with unlimited threading; no difference between posts, threads, or forums. Any post can be answered an unlimited amount of time, and the same post can have several children, themselves having

  • How do we handle Categories in database schema and model classes

    How do we handle Categories in database schema and model classesOctober 1

    I am trying to develop an ecommerce solution but stuck on the categories management issue and don't know how to proceed further. See, A category will have a parent category, which in-return can also point to another parent and so on, until the root c

  • Are there any free online collaboration database schema tools? October 31

    I am interested in a free and simple MySQL tool which can offer online collaboration functionality in creating database general schema. Do you know any? --------------Solutions------------- You can use Vertabelo. Vertabelo is an online database desig

  • looking for a complete documentation for Drupal's database schema - need to have some overview

    looking for a complete documentation for Drupal's database schema - need to have some overviewJanuary 30

    well dear buddies, i am pretty new to drupal. Just set up a drupal commons now i 'd like to add some data to my drupal, and i dont really understand the database model/scheme. Can somebody point me to some documentation? For example i need to underst

  • Handling database schema changes when pushing new versionsApril 27

    During times of heavy development, the database schema changes both rapidly and continuously, and by the time our weekly push to the beta build comes around, the schema has changed so much that the only sensible option is to nuke all of the tables I

  • Which database schema should I pick?November 22

    I am planning a database scheme and I hesitate between two designs, which one should I choose? For the purpose of the question, assume we want to prepare a database scheme for an application managing student information in a university. There is seve

  • SharePoint 2010 Database Schema Versions Not Responding errorFebruary 2

    We have multiple site collections in one webapplication and each site collection is in its own content DB. Now when I browse to one of the site collection, it gives me HTTP 404 not found error. And When I dig deeper what's causing it, I found below e

  • Load The Drupal Database Schema in PostgreSQL

    Load The Drupal Database Schema in PostgreSQLMarch 17

    I was following the tutorial in the link below to install Drupal 7 with PostgreSQL. http://pgedit.com/install_drupal I'm stuck with step 3. 3.Load the Drupal database schema Your Drupal installation has a top level folder called 'database' where you

  • Ideas for Database schema for Automotive web appMay 26

    For the last couple of days I have been searching all over Google for database schema for my automotive web application [like a marketplace of sort] (in Laravel 5 PHP). I have tried many models some I don't even remember their names. I have tried EAV

  • Need help making a sane database schemaNovember 26

    I am writing a backend for an application that has 4 main types of content: a message, a post, a thread, and a thread reply. Right now, these are all stored in separate MySQL tables: a table for each type of content. I would like to introduce a moder

  • Different field name for database / schemaJanuary 26

    TL:DR; is it possible to have a field foobarbaz which is stored with a different name f in the document? I'm currently planning a new database schema which will contain a lot of documents (expecting something between 10 and 100k documents power day).

  • Data Mapping between two different Database SchemasJanuary 27

    I have to design a system with an sql database whose work is to get data from different databases which may be in another database such as mysql or oracle etc. Then the system will map the attributes of that database with my database schema and store

  • Database schema for a products and bugs aggregatorJanuary 29

    As an exercise in learning, I'm creating a bugs/issues tracking and "aggregator" web application. A simple app consisting of products, issues, reviews and links related to it. The links will be urls pointing to issues/reviews already created on

  • Easy Database Schema Diagrams with DBVisualizer

    Easy Database Schema Diagrams with DBVisualizerMay 14

    I've never particularly liked databases. Don't get me wrong–I love using them. You know, filling them with data, watching my users fill them with data, deleting my users' data* - but as long as it's all from the safety of a server-side programming la

  • What is purpose of database schema? April 20

    What is the purpose of database schema? Where can I find more information about this? It's not table, it's not database, what is it? --------------Solutions------------- From the quackit.com tutorial – SQL Server - Database Schemas: A database schema

  • How to create a new database schema in SQL Express and give access to any user account of the current machine?July 30

    Mixed Mode / SQL Authentication is disabled. Only Windows Authentication is available. A database/schema has been created in the default instance (.\SQLEXPRESS) and is named "MyDatabase". Using a SQL script that will be run by an administrator u

Copyright (C) 2018 ceus-now.com, All Rights Reserved. webmaster#ceus-now.com 14 q. 0.570 s.