Homemade data logging with Arduino ?

Post here information about your own engineering projects, including but not limited to building your own car or designing a virtual car through CAD.
woohoo
1
Joined: Sun Aug 10, 2008 12:12 am

Re: Homemade data logging with Arduino ?

Post by woohoo » Tue Sep 13, 2016 11:02 am

Jef Patat wrote:Sorry, I ment Due: https://www.arduino.cc/en/Main/ArduinoBoardDue

There are so many boards these days that choosing one becomes an art in itself.
Thanks Jeff! I think this is the board that is replaced by the Mega 2560 ? We have the 2560 for use :)
(and a UNO for backup / comparison...)
The only way to close a stupid question is to give a smart answer

johnny99
5
Joined: Thu Apr 09, 2009 6:28 pm
Location: Killucan Westmeath Ireland

Re: Homemade data logging with Arduino ?

Post by johnny99 » Tue Sep 13, 2016 12:14 pm


nzjrs
49
Joined: Wed Jan 07, 2015 10:21 am
Location: Austria

Re: Homemade data logging with Arduino ?

Post by nzjrs » Wed Sep 14, 2016 12:37 pm

Arduino are not serious microcontrollers, nor a serious organisation. Their stadard library is poor quality. Arduino are playdough for engineers.

Take a ARM cortex, stm32, mBed or something.

graham.reeds
21
Joined: Thu Jul 30, 2015 8:16 am

Re: Homemade data logging with Arduino ?

Post by graham.reeds » Wed Sep 14, 2016 2:32 pm

nzjrs wrote:Arduino are not serious microcontrollers, nor a serious organisation. Their stadard library is poor quality. Arduino are playdough for engineers.

Take a ARM cortex, stm32, mBed or something.
Horses for courses.

If they were thinking of making this a professional logger with sales and support, etc. then you might have a point.

As a learning exercise then it is perfectly suited.

I once looked at making a data logger for my kart. The estimated cost was about £400 but would have the capabilities far beyond what commercial ones could do for twice the price. However I had to make it and I would have had to implement the peripherals also. In the end it was easier to buy one rather than make my own so I will be watching with interest.

johnny99
5
Joined: Thu Apr 09, 2009 6:28 pm
Location: Killucan Westmeath Ireland

Re: Homemade data logging with Arduino ?

Post by johnny99 » Wed Sep 14, 2016 11:19 pm

Thanks graham.reed. It all started with a chat and we ran with it. Why not, even if it fails, we had fun and learned a huge amount. total budget so far is under 100 Euro and many interesting Saturday afternoon's. If we get something working and it leads woohoo improving his lap times, bonus. If the basics work we can move to better equipment

John

Brian Coat
98
Joined: Sat Jun 16, 2012 5:42 pm

Re: Homemade data logging with Arduino ?

Post by Brian Coat » Wed Sep 14, 2016 11:49 pm

I know the Arduino has built in ADC but maybe the Raspberry Pi could do this well if mated to a good ADC chip.

People on here said the RPi is not up to the job but I've seen it used for very resource-intensive DSP work - e.g. active noise cancellation. That is some serious real-time stuff.

It is a serious device and there is a massive expert community.

You can get all kinds off add-ons for Pi like ADC and i/o CPLD to help with data logging at very high speeds.

Not saying Arduino is not up to it, though.

johnny99
5
Joined: Thu Apr 09, 2009 6:28 pm
Location: Killucan Westmeath Ireland

Re: Homemade data logging with Arduino ?

Post by johnny99 » Thu Sep 15, 2016 12:22 am

Thanks Brian, from checking out what had the best ability and easier to use, we choose the Arduino, we may be wrong, who knows yet. Massive support for the Arduino too, let's see what happens.

John

VL400
0
Joined: Thu Sep 15, 2016 12:41 am

Re: Homemade data logging with Arduino ?

Post by VL400 » Thu Sep 15, 2016 12:52 am

I have used two arduino based boards in vehicle data loggers, first one used a teensy 2++ and the current one a teensy 3.2. Both have a whole bunch of analog and digital inputs, and an interface to the cars data bus. The teensy 3.2 also has a serial data interface for connection to other modules not on the bus. All analog and digital samples are then added to the vehicle data to form a larger data frame for playback on the PC software TunerPro RT. I also use this for calibration adjustments on various modules in the car, so the arduino data logger records to an SD card in the file format for that software.

I am not recording at the frame rates you are proposing due to the vehicle data being comparatively slow, but am sampling in the KHz range and then using adjustable filter coeffs for fast and slow moving sensors.

One thing to note is the SD library for the arduino is fairly slow to write, you may find gaps in your sampling during a write.

I can provide more info, but you are on the right track :)

nzjrs
49
Joined: Wed Jan 07, 2015 10:21 am
Location: Austria

Re: Homemade data logging with Arduino ?

Post by nzjrs » Thu Sep 15, 2016 7:56 am

I'll also second the teensy recommendation.

Jef Patat
92
Joined: Fri May 06, 2011 1:40 pm

Re: Homemade data logging with Arduino ?

Post by Jef Patat » Thu Sep 15, 2016 8:55 am

Brian Coat wrote:I know the Arduino has built in ADC but maybe the Raspberry Pi could do this well if mated to a good ADC chip.

People on here said the RPi is not up to the job but I've seen it used for very resource-intensive DSP work - e.g. active noise cancellation. That is some serious real-time stuff.

It is a serious device and there is a massive expert community.

You can get all kinds off add-ons for Pi like ADC and i/o CPLD to help with data logging at very high speeds.

Not saying Arduino is not up to it, though.
There's a basic difference between arduino (and the likes) and the Pi (and the likes). The Pi runs an OS and the micros do not. For serious datalogging you need as little jitter as possible, of course that is dependent on the task at work, but still, Linux is not meant to be realtime. Certain amounts of jitter makes some postprocessing / correct interpretation impossible.

I work for a company that makes test systems for the off road vehicle industry, hard realtime is everything to us. We use EtherCAT with distributed clocks which allows us microsecond jitter. This makes Fourrier analysis possible.

I didn't mean to say one is better than the other, it depends on so many factors. I just wanted to say this might be an argument in the decision process.

Brian Coat
98
Joined: Sat Jun 16, 2012 5:42 pm

Re: Homemade data logging with Arduino ?

Post by Brian Coat » Thu Sep 15, 2016 9:09 am

Thanks, Jef.

I think these are very good points.

The Pi runs whatever you load onto it and you are so right about the standard OS being slow.

I forgot to mention that the active noise cancellation application using RPi does not use Linux OS. It just uses compiled C code, including the standard libraries which are available for the hardware interfaces etc.

And it is done this was because the OS is too slow for this very high speed application.

Sorry, an important omission, that.
Last edited by Brian Coat on Thu Sep 15, 2016 10:08 am, edited 1 time in total.

Jef Patat
92
Joined: Fri May 06, 2011 1:40 pm

Re: Homemade data logging with Arduino ?

Post by Jef Patat » Thu Sep 15, 2016 10:08 am

Are you talking bare metal programming now? The GPU boot loader takes care of starting the system to end with loading from the SD card, so with the correct knowledge and experience you can run 'whatever'. But that's not the intent of the platform, as such you will have minimal community support. I don't think the TS has this kind of competences.

Brian Coat
98
Joined: Sat Jun 16, 2012 5:42 pm

Re: Homemade data logging with Arduino ?

Post by Brian Coat » Thu Sep 15, 2016 10:13 am

Yes I am. But there are examples of pretty fast data logging using Raspi+Linux, albeit with other hardware like CPLD to unburden the PI+OS a bit.

I think that is a long winded way of me saying "you are right" (crawls back into box)!

woohoo
1
Joined: Sun Aug 10, 2008 12:12 am

Re: Homemade data logging with Arduino ?

Post by woohoo » Thu Sep 22, 2016 9:54 pm

HI,
So, to give an update I have been playing with a Arduino Uno, and have gotten it to do the basics, read analog inputs and spit them into a CSV file. So far so good.
Using very simplistic C++from IDE (the native Arduino language) I was able to get it to extract info at an average of 1.5 milliseconds (ca. 500Hertz)
So far from the desired 2K but not.. hopefully.. not a million miles away. I have not yet done any speed optimizations, so there is, again, hopefully some space there to speed up the data collection.

Now then, in order to get the date to work with anything, I have the following question:

Does anyone know the CSV order for dlog99 ?
Our ideal software, GEMS opens dlog99 files, which in turn opens CSV files. These however have to be formatted a specific way.
Not having a car with a logging system using dlog99 / GEMS means it is difficult to second guess the formatting.

But, any other solution is also welcome :)
The only way to close a stupid question is to give a smart answer

Tim.Wright
435
User avatar
Joined: Fri Feb 13, 2009 5:29 am

Re: Homemade data logging with Arduino ?

Post by Tim.Wright » Fri Sep 23, 2016 9:29 am

Why do you need a sampling rate of 2khz? Do you even have a sensor that is responsive at such a high frequency?
Not the engineer at Force India