Periodic data append to Nanocubes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Periodic data append to Nanocubes

Ranjith
Hi Everyone,

I am re-posting this question about data append, to get a clear response whether this is possible with existing Nanocubes or not .  

The scenario is as below:
1. Example, At 9ET I have one set of records (assume 1000) that I want to push to NC Server. I am able to do that without any issues, since this the first time i am starting the NC server with the data load (1000 records).
2. At 10 ET I got 100 more records that I need to append to the NC Server, without stopping the NC Server or webserver that are already running as users are using the GUI.
3. At 11 ET few more records to append similar to above in #2 and so on as I will be getting more data every hour.

Is this type of append possible in existing nanocubes implementation without restarting ncserver(with complete data every time) ??

Thanks,
Ranjith
Reply | Threaded
Open this post in threaded view
|

Re: [Nanocubes-discuss] Periodic data append to Nanocubes

salivian
Hi,

The nanocube server reads data from stdin continuously (specify the csv file as '-').

eg. 
$StreamingDataProgram | python csv2Nanocube.py --latcol=latitude --loncol=longitude --timecol=time --catcol=device,make - | ncserve ...
 

Please try to connect the data source with nanocubes directly, so the nanocube server will read the new data every hour.  

If your data is delivered to some files that resides in a directory, I suggest that you write a script to check for new files, read the new file in then print the new file to the nanocubes server.

If you use python this library may help you for files checking.


Horace



On Tue, Sep 16, 2014 at 12:32 PM, Ranjith <[hidden email]> wrote:
Hi Everyone,

I am re-posting this question about data append, to get a clear response
whether this is possible with existing Nanocubes or not .

The scenario is as below:
1. Example, At 9ET I have one set of records (assume 1000) that I want to
push to NC Server. I am able to do that without any issues, since this the
first time i am starting the NC server with the data load (1000 records).
2. At 10 ET I got 100 more records that I need to append to the NC Server,
without stopping the NC Server or webserver that are already running as
users are using the GUI.
3. At 11 ET few more records to append similar to above in #2 and so on as I
will be getting more data every hour.

Is this type of append possible in existing nanocubes implementation without
restarting ncserver(with complete data every time) ??

Thanks,
Ranjith



--
View this message in context: http://nanocubes-discuss.64146.x6.nabble.com/Periodic-data-append-to-Nanocubes-tp118.html
Sent from the nanocubes-discuss mailing list archive at Nabble.com.

_______________________________________________
Nanocubes-discuss mailing list
[hidden email]
http://mailman.nanocubes.net/mailman/listinfo/nanocubes-discuss_mailman.nanocubes.net


_______________________________________________
Nanocubes-discuss mailing list
[hidden email]
http://mailman.nanocubes.net/mailman/listinfo/nanocubes-discuss_mailman.nanocubes.net