Page 2 of 3
Re: Homemade data logging with Arduino ?
Posted: 13 Sep 2016, 12:02
by woohoo
Thanks Jeff! I think this is the board that is replaced by the Mega 2560 ? We have the 2560 for use

(and a UNO for backup / comparison...)
Re: Homemade data logging with Arduino ?
Posted: 13 Sep 2016, 13:14
by johnny99
Re: Homemade data logging with Arduino ?
Posted: 14 Sep 2016, 13:37
by nzjrs
Arduino are not serious microcontrollers, nor a serious organisation. Their stadard library is poor quality. Arduino are playdough for engineers.
Take a ARM cortex, stm32, mBed or something.
Re: Homemade data logging with Arduino ?
Posted: 14 Sep 2016, 15:32
by graham.reeds
nzjrs wrote:Arduino are not serious microcontrollers, nor a serious organisation. Their stadard library is poor quality. Arduino are playdough for engineers.
Take a ARM cortex, stm32, mBed or something.
Horses for courses.
If they were thinking of making this a professional logger with sales and support, etc. then you might have a point.
As a learning exercise then it is perfectly suited.
I once looked at making a data logger for my kart. The estimated cost was about £400 but would have the capabilities far beyond what commercial ones could do for twice the price. However I had to make it and I would have had to implement the peripherals also. In the end it was easier to buy one rather than make my own so I will be watching with interest.
Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 00:19
by johnny99
Thanks graham.reed. It all started with a chat and we ran with it. Why not, even if it fails, we had fun and learned a huge amount. total budget so far is under 100 Euro and many interesting Saturday afternoon's. If we get something working and it leads woohoo improving his lap times, bonus. If the basics work we can move to better equipment
John
Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 00:49
by Brian Coat
I know the Arduino has built in ADC but maybe the Raspberry Pi could do this well if mated to a good ADC chip.
People on here said the RPi is not up to the job but I've seen it used for very resource-intensive DSP work - e.g. active noise cancellation. That is some serious real-time stuff.
It is a serious device and there is a massive expert community.
You can get all kinds off add-ons for Pi like ADC and i/o CPLD to help with data logging at very high speeds.
Not saying Arduino is not up to it, though.
Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 01:22
by johnny99
Thanks Brian, from checking out what had the best ability and easier to use, we choose the Arduino, we may be wrong, who knows yet. Massive support for the Arduino too, let's see what happens.
John
Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 01:52
by VL400
I have used two arduino based boards in vehicle data loggers, first one used a teensy 2++ and the current one a teensy 3.2. Both have a whole bunch of analog and digital inputs, and an interface to the cars data bus. The teensy 3.2 also has a serial data interface for connection to other modules not on the bus. All analog and digital samples are then added to the vehicle data to form a larger data frame for playback on the PC software TunerPro RT. I also use this for calibration adjustments on various modules in the car, so the arduino data logger records to an SD card in the file format for that software.
I am not recording at the frame rates you are proposing due to the vehicle data being comparatively slow, but am sampling in the KHz range and then using adjustable filter coeffs for fast and slow moving sensors.
One thing to note is the SD library for the arduino is fairly slow to write, you may find gaps in your sampling during a write.
I can provide more info, but you are on the right track

Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 08:56
by nzjrs
I'll also second the teensy recommendation.
Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 09:55
by Jef Patat
Brian Coat wrote:I know the Arduino has built in ADC but maybe the Raspberry Pi could do this well if mated to a good ADC chip.
People on here said the RPi is not up to the job but I've seen it used for very resource-intensive DSP work - e.g. active noise cancellation. That is some serious real-time stuff.
It is a serious device and there is a massive expert community.
You can get all kinds off add-ons for Pi like ADC and i/o CPLD to help with data logging at very high speeds.
Not saying Arduino is not up to it, though.
There's a basic difference between arduino (and the likes) and the Pi (and the likes). The Pi runs an OS and the micros do not. For serious datalogging you need as little jitter as possible, of course that is dependent on the task at work, but still, Linux is not meant to be realtime. Certain amounts of jitter makes some postprocessing / correct interpretation impossible.
I work for a company that makes test systems for the off road vehicle industry, hard realtime is everything to us. We use EtherCAT with distributed clocks which allows us microsecond jitter. This makes Fourrier analysis possible.
I didn't mean to say one is better than the other, it depends on so many factors. I just wanted to say this might be an argument in the decision process.
Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 10:09
by Brian Coat
Thanks, Jef.
I think these are very good points.
The Pi runs whatever you load onto it and you are so right about the standard OS being slow.
I forgot to mention that the active noise cancellation application using RPi does not use Linux OS. It just uses compiled C code, including the standard libraries which are available for the hardware interfaces etc.
And it is done this was because the OS is too slow for this very high speed application.
Sorry, an important omission, that.
Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 11:08
by Jef Patat
Are you talking bare metal programming now? The GPU boot loader takes care of starting the system to end with loading from the SD card, so with the correct knowledge and experience you can run 'whatever'. But that's not the intent of the platform, as such you will have minimal community support. I don't think the TS has this kind of competences.
Re: Homemade data logging with Arduino ?
Posted: 15 Sep 2016, 11:13
by Brian Coat
Yes I am. But there are examples of pretty fast data logging using Raspi+Linux, albeit with other hardware like CPLD to unburden the PI+OS a bit.
I think that is a long winded way of me saying "you are right" (crawls back into box)!
Re: Homemade data logging with Arduino ?
Posted: 22 Sep 2016, 22:54
by woohoo
HI,
So, to give an update I have been playing with a Arduino Uno, and have gotten it to do the basics, read analog inputs and spit them into a CSV file. So far so good.
Using very simplistic C++from IDE (the native Arduino language) I was able to get it to extract info at an average of 1.5 milliseconds (ca. 500Hertz)
So far from the desired 2K but not.. hopefully.. not a million miles away. I have not yet done any speed optimizations, so there is, again, hopefully some space there to speed up the data collection.
Now then, in order to get the date to work with anything, I have the following question:
Does anyone know the CSV order for dlog99 ?
Our ideal software, GEMS opens dlog99 files, which in turn opens CSV files. These however have to be formatted a specific way.
Not having a car with a logging system using dlog99 / GEMS means it is difficult to second guess the formatting.
But, any other solution is also welcome

Re: Homemade data logging with Arduino ?
Posted: 23 Sep 2016, 10:29
by Tim.Wright
Why do you need a sampling rate of 2khz? Do you even have a sensor that is responsive at such a high frequency?