Homemade data logging with Arduino ?

Post here information about your own engineering projects, including but not limited to building your own car or designing a virtual car through CAD.
VL400
0
Joined: Thu Sep 15, 2016 12:41 am

Re: Homemade data logging with Arduino ?

Post by VL400 » Thu Sep 29, 2016 1:03 am

If you havnt got the dlog99 format sussed yet woohoo, flick me a PM with your email address and can send you some info as I also use this software (I export files from a program TunerPro as CSV and then have a macro to convert to dlog99 for GEMS)

Also, another thing to add to your logger is a MPU6050 board for gyro and accelerometer. Have in the past 12 months or so started using these instead of some other types and they work really well.

woohoo
1
Joined: Sun Aug 10, 2008 12:12 am

Re: Homemade data logging with Arduino ?

Post by woohoo » Sun Oct 02, 2016 10:55 pm

HI VL400 thank very much for that offer!
I have not yet been able to get any accurate data files from dlog99 and would love to see how the program expects its values to be :)

Thanks for the suggestion on the G-Meter as well!
The only way to close a stupid question is to give a smart answer

MadMatt
147
Joined: Sat Jan 08, 2011 3:04 pm

Re: Homemade data logging with Arduino ?

Post by MadMatt » Mon Oct 03, 2016 8:22 pm

Interesting topic, as I have also just started to learn Arduino (I have the Mega2560). My aim is also to log analog inputs, process the signals and display them on a simple LCD screen, save them on an SD card (CSV format), and post-process them in Matlab.

I'm looking for a accelerometer shield and a data logging shield that won't eat up all the analog input pins tho!

Greg Locock
165
Joined: Fri Jun 29, 2012 11:48 pm

Re: Homemade data logging with Arduino ?

Post by Greg Locock » Tue Oct 04, 2016 4:05 am

I've been buying my sensors on little PCBs with only 6 pins or so, not shields. Shields seem to be a solution looking for a problem, given they cost more than cloned cpus.

MadMatt
147
Joined: Sat Jan 08, 2011 3:04 pm

Re: Homemade data logging with Arduino ?

Post by MadMatt » Tue Oct 04, 2016 12:46 pm

It would make sense yes, but then one has to consider packaging, therefore creating mounts and brackets to hold the sensors. Not the end of the world, but not everybody has the ability to create these parts.

Woohoo, what way of data logging did you use? SD card? Eeprom? Server?

I believe if you want to log at 100Hz (which is more than enough for most automotive/motorsport activities), the SD card option sounds plausible. Am I correct?

woohoo
1
Joined: Sun Aug 10, 2008 12:12 am

Re: Homemade data logging with Arduino ?

Post by woohoo » Thu Oct 06, 2016 10:26 am

HI Guys,

Shields are good for their purpose, which is easy packaging and snap together functionality. The only problem with them, is they are made for the Arduino Uno's, not the bigger boards we would tend to use.

The shield I have is this:
https://learn.adafruit.com/adafruit-dat ... d/overview
Which both has a SD card and a real time clock.
There is a new version which also has a GPS device, but I do not have it.
https://www.adafruit.com/products/1272

If you are concerned about packaging, you could solder an accelerometer module on top of the shield, something like this:
https://www.sparkfun.com/products/9269
But still, you have to be mindful of how the connectors line up...

If you give me a day or so, I can publish the code I use and the writing times onto SD cards.

Where I myself am going, is to get a Teensy for the suspension sampling at the very least, use the 2560 for the rest of analog sampling, and get both devices to send data to a Raspberry Pi, which will collect all this through serial connection using a python script, and write to a USB stick there. There are a of questions on that still though...

The reason for that, is that the Arduino is not quick enough to sample at high resolution, so it needs a Teensy, which does not have the capability to write to anything, and can only send data over serial, and even if the Arduino was fast enough, it still does not not have multitasking, so it cannot read analog and write at the same time.
The only way to close a stupid question is to give a smart answer

MadMatt
147
Joined: Sat Jan 08, 2011 3:04 pm

Re: Homemade data logging with Arduino ?

Post by MadMatt » Thu Oct 06, 2016 11:46 am

Thank you for your input! Yes, it can be a real pain in the ass to have to deal with all these different platforms.

But as it was discussed, do you really need >1kHz of sampling rate? From my experience 250Hz is enough for suspension work. For me the limitation of the Mega is its 10bits resolution. I am considering the following shield to extend the resolution:

http://mayhewlabs.com/products/extended-adc-shield

This improves 8 channels, with a sampling rate up to 100kHz and higher resolution. Then your SD shield could be used on top of it to really make it a nice little logging device!

woohoo
1
Joined: Sun Aug 10, 2008 12:12 am

Re: Homemade data logging with Arduino ?

Post by woohoo » Thu Oct 06, 2016 2:17 pm

Thanks, I did not know if that board, that is worth a bit of research.

If you do not have a Arduino yet, get the Due one, it has a faster chip, and allows for higher sample rate.
The only way to close a stupid question is to give a smart answer

MadMatt
147
Joined: Sat Jan 08, 2011 3:04 pm

Re: Homemade data logging with Arduino ?

Post by MadMatt » Thu Oct 06, 2016 6:25 pm

Well, it looks interesting, but CAREFUL! Only 3.3V supported on the analog inputs, which makes it useless for automotive applications (unless you want to do some extensive signal management)! I hate those limitations on either Arduino or Raspberry. I understand they want to cut costs, but come on!

Caito
13
Joined: Tue Jun 16, 2009 4:30 am
Location: Switzerland

Re: Homemade data logging with Arduino ?

Post by Caito » Wed Oct 19, 2016 5:22 pm

Hey guys!

Just saw this post, I decided to jump in. I've done my own data logger system as a thesis to graduate, you can read some here http://www.f1technical.net/forum/viewto ... 14&t=14890.


First some clarification on the sampling frequency for the damper measurement. It's true that if the signal is 25Hz then you only need a 50Hz sampling rate. But that is theory only, because if you do that you will have aliasing. So you need to put some low-pass filter to avoid this, an ideal filter is not possible (not causal, you need information from the future). And you don't want to mess with complicated filters, just an RC, order 2 filter at most. In order for the filter not to affect your signal you put your cutoff signal at 50/100Hz, and then sample at 1kHz to make sure you have attenuated the noise enough.

Of course a 1kHz signal is ussually useless from a visualization point of view, just think of the most basic example. You will be watchcing 10 seconds of laptime on an HD screen (ca. 2000 pixels wide). You have 10.000 points and can only draw 2.000. What you do with your 1kHz signal is downsample and filter it, basically just get some points and average them, so you can have a signal with less noise. With this, 10bit should be more than enough, assuming you get 8 useful and effective bits out of the Arduino, for 80mm of travel this is 0.3mm/bit, even more interesting, since you oversampled you would get a little "extra bits" (one bit per every 4x oversample).

Going into all the Arduino stuff. Arduino is useful for hobbyists and DIY, but it's extremely inefficient. So try to get the fastest arduino you can, some people mentioned the Teensy and it would seem like a good option.

Try to start with the most basic sensors, let's say GPS, accel, gyro, and a couple analog (throttle/break/wheel). You need to synchronize all of these sensors, use the fastest one to create periodic interrupt.
Lets imagine:
ADC 1kHz, GPS 10HZ, accel/gyro 100Hz.

Make a periodic interrupt at 1ms. Every time you enter this interrupt trigger the ADC (but don't wait for the conversion to end, there's no time for that), and get the conversion results from the last time. Every 10 times you access the accel/gyro, and every 100 times you access the GPS. So now you have everything synchronized on the same timebase.

Wait until you have a chunk of data before putting it to the SD card. Writing mere bytes into the SD is really inefficient. Also, try not to use .csv for the SD, just write your plain bytes. If you use csv you need to convert the values to string, do processing, and require many times more data. You just want the arduino to capture and save, you have infinite time in your computer to process. If you can't manage to do it with an SD card, you can use a simple memory chip (32MB will probably be more than 1 hour of data), and then upload the data to the computer.

Once you finished logging, you can download your data and use all your CPU power to turn that into a csv file, format it, make it pretty, and then show it with whatever soft you like.

You could put a real time linux to a raspberry pi and use that, but it really doesn't make much sense IMHO, that is made for real time processing and you really are not doing processing. Just managing and scheduling some peripherals.

And for the 3.3V, that is absolutely no problem, just an extra voltage divider, and since you were going to put an RC filter anyways, it's only 1 more resistor.

Hope it helps a little bit. Just let me know if you have any particular doubts.
Come back 747, we miss you!!

SameSame
13
Joined: Thu Jun 16, 2016 5:44 pm

Re: Homemade data logging with Arduino ?

Post by SameSame » Wed Oct 19, 2016 6:53 pm

Caito wrote:Hey guys!...
Excellent stuff! +1!

woohoo
1
Joined: Sun Aug 10, 2008 12:12 am

Re: Homemade data logging with Arduino ?

Post by woohoo » Mon Nov 21, 2016 10:37 pm

Caito wrote:Hey guys!

Just saw this post, I decided to jump in. I've done my own data logger system as a thesis to graduate, you can read some here http://www.f1technical.net/forum/viewto ... 14&t=14890.
....
That is great stuff Caito, many hanks for posting this, and your earlier thread which is extremely interesting.

I can see the limitations of the Arduino, (and we will probably upgrade to a Teensy soon) but I am working away on the Ardino, simply for the sake that it is one thing to get the data from a sensor to a device, but to get the data into a valid format for a piece of software is another.

I am still working on second guessing GEMS and its data. Though I am getting better, have mapping etc. done :)
The only way to close a stupid question is to give a smart answer

Zynerji
61
Joined: Wed Jan 27, 2016 3:14 pm

Re: Homemade data logging with Arduino ?

Post by Zynerji » Tue Nov 22, 2016 4:37 am

I wonder if this is worth checking out. The windows 10 arduino with built in WiFi might just be the answer.

http://www.lattepanda.com

Caito
13
Joined: Tue Jun 16, 2009 4:30 am
Location: Switzerland

Re: Homemade data logging with Arduino ?

Post by Caito » Tue Nov 22, 2016 5:41 pm

woohoo wrote: ...

I am still working on second guessing GEMS and its data. Though I am getting better, have mapping etc. done :)

You can buy a sort of license from Motec to run your own data on their soft, but it can get quite expensive, and it's on a yearly basis if I'm not mistaken.

Otherwise, I think this guys might have some sort of open-source soft. (I don't know, just googled them)
https://www.autosportlabs.com/keeping-promises/
Come back 747, we miss you!!

daftKow
0
Joined: Fri Oct 13, 2017 8:14 pm

Re: Homemade data logging with Arduino ?

Post by daftKow » Fri Oct 13, 2017 8:49 pm

Hello All,

I've stumbled on the thread while looking for information on a small project on my own; trying to create a simple (GPS+IMU) logger for my car, to learn about vehicle dynamics and using that data to improve myself as a driver.

I'd love any information regarding CSV -> dlog99 -> GEMS (I tried sending you a PM, VL400; but it wont let me until I have 5 posts).

I've been recommended the Venus 838FLPx for GPS (running at 20Hz down from 50Hz to reduce the noise) and IMU6050 from above. I just need an Arduino or Raspberry Pi to write that data in CSV to a USB/microSD card so I can take a look using GEMS between laps.

Any advice would be appreciated!

Regards,