STM CircleOS forum / SDIO+FAT speed

kubsztal · 2009-03-08 15:45:33

Hi All,

as this is my 1st post here I would like to say HELLO to everybody and wish all of you lot of fun during playing with your Primer

Anyway, I've made few writing tests of SD card (SanDisk Mobile Ultra 2GB) and here are the results (MCU speed set to 72MHz, internal transfer):
- 1 sector (200B or 400B): ~11ms max. 13ms
- 4 sectors (2000B): ~16ms max. 26ms
- 8 sectors (4000B): ~22ms max. 28ms
- 16 sectors (8000B): ~35ms to 60ms
- 32 sectors (1600B): ~66ms ~240kB/s (similar to transfer via USB - max. 225kB/s).

As I can find in the sources of CircleOS - SDIO is set to 24MHz/1 wire communication but in case of USB transfer, SDIO communication is set to 4 wires (in both cases DMA is enabled I guess).

My question: is it possible to make SD writing more efficient? According to following tests (ChaN's ffat) ATMega64 clocked 9.2MHz has similar writing speed:
http://elm-chan.org/fsw/ff/img/rwtest.png
and LPC2368/72MHz with MMC via MCI is much faster:
http://elm-chan.org/fsw/ff/img/rwtest2.png

Or maybe DOSFS system is so slow and ChaN's ffat is so efficient?

Last edited by kubsztal (2009-03-08 17:09:22)

Francis · 2009-03-09 07:16:15

Just a short notice: by default the CPU clock is set to 36MHz. The user can modify it in the Config|Power menu. Are you numbers given for the max frequency, or the default frequency?

The integration of the FS+USB-MS has been made by Sjoerd, but I understood that the first goal was having something that works. I believe that there was no optimization up to now... As you wrote, if a single-wire is used instead of 4 wires, it could be optimized.. If you want to propose an update, we would integrate your changes in the next release.

sjoerd · 2009-03-09 07:23:53

That's correct. My main focus was to get at least something with FAT/SDIO working on the Primer2. I'm pretty sure some things can be optimized for speed and perhaps code size as well.

Everybody is free to improve the current implementation and share it with the community

kubsztal · 2009-03-09 16:39:11

I missed the fact that DFS_Mount reinitializes transfer to 4 wires, so there are two places for optimization: fat file system and/or sdio driver - later I will try to figure out what could be done here. MCU clock was set to max. (72MHz) for these measurements, I haven't checked other frequencies yet.

Francis · 2009-03-09 17:32:23

Perhaps the DMA 'memory to memory mode' could be used for DFS_memcpy. Don't know whether we spend time in this function.

jetcode · 2009-05-05 03:39:01

If anyone is interested ...
I have generated a version of DFS that features the following:

1) Added multiple sector read/write capability to DFS using legacy API.
The DFS_FileRead and DFS_FileWrite functions were modified to transfer multiple sectors in one instance via modified DFS_SectorRead and DFS_SectorWrite functions. This is useful because the call overhead is now 1 / nsectors and that translates to an increase in throughput efficiency.

2) Isolated file length update to directory to the function DFS_Flush().
This permits the end user to choose when to update the file length in the directory rather than after each data sector is written (file length update requires sector read/write cycle). This strategy is similar to DOS based FAT where the user calls the flush command to update the file directory (file length) and when the file is closed DFS_CloseFile. It's more to manage but you can optionally flush every 'n' sectors. I also hard coded a flush to occur on cluster boundaries. I violated the original design by adding a field in FILEINFO for keeping track of whether the file length needs to be updated in the directory. Only DFS_WriteFile calls enable the file to be flushed. You may not agree with this approach and can easily restore the code back to its original functionality.

3) FAT12 removed
As far as I know there are no micro SD cards which format to FAT12.

4) Rewrote DFS_memcpy and other functions that appear to be less efficient then they could be. That's my take of course.

I'm just beginning to test the code but the changes are rather simple and you are welcome to take a look as it is now.

Joe

Last edited by jetcode (2009-05-07 00:31:41)

repzak · 2009-05-17 00:35:07

Hello,

I could be interasted, do you have the complete circle_OS project working?

Kasper

Francis · 2009-05-17 01:26:27

It would be nice to publish these files. We would integrate them into CircleOS. It seems that several persons complain about the first version..

jetcode · 2009-05-17 15:28:29

I basically copied the DFS and renamed it FDFS which compiles but hasn't been tested yet. I am working on design specifications so it may be a while before I get to test. I'll send the files by email and you can integrate them and/or evaluate the changes made.

I sent an email to Francis with the files as they are. I will be testing sometime soon but I am in the middle of a design specification right now.

sjoerd,
I think you did a great job putting the mass storage system together. I think this platform is rich with features and the fact that I can leverage it into my own products is worth a small fortune in my book.

Last edited by jetcode (2009-05-17 16:33:23)

Francis · 2009-05-18 06:45:07

If someone tests them, and thinks that the integration within V3.7 is correct, we will publish the files. But our side, we will not have time for testing in the next few weeks..

repzak · 2009-05-18 07:07:54

Hello,

Let me get a copy, then i will try them out, maybe they can help on my other problems..

Kasper

jetcode · 2009-05-18 17:22:13

Kaspar,

I sent you an email. When you reply I will have your email address and can send you the files. I am testing the changes I made and known issues.

Joe

sjoerd · 2009-05-19 16:20:23

Good to see that development continues on the FS area.

My SDIO+FS implementation/integration hasn't been much more then taking the SDIO lib from ST and combine it with the standard DFS. If anybody can fix problems or improve it in any kind of way, that's just great!

Unfortunatly, I cannot spend much time on it the near future so all input/updates are welcome.

jetcode · 2009-05-19 17:26:48

I'm not sure I will be able to add anything of value. I will spend some time seeing if I can get multi-block read/write to work.

zb7401 · 2009-05-20 06:57:58

good news also want to use DOSFS。waiting

jetcode · 2009-05-20 15:45:47

You can use DOSFS right now - Do not wait for what I am trying to do - I've spent 2 days so far and have made some progress but do not have a working solution and may not be able to achieve one with the time I have to give to this project at this time.

The files I sent Francis were untested and unusable.

Francis · 2009-05-20 16:15:44

Thanks for the information !

jetcode · 2009-05-20 16:52:51

Francis what is the strategy to test changes to the OS? Where is test code placed for debugging an OS service?

How do I get access to the low level OS functions like the SD card library from within my application? Without including a version of the SDcard source in my project the function stubs are undefined. If I include a version of SD card in my code the interrupt handler is calling the SD card code in the OS not the version I have. I have tried to replace the ISR handler for the SD card code but it didn't work and that may be my fault.

Last edited by jetcode (2009-05-20 16:57:52)

jetcode · 2009-05-20 17:52:23

I am taking the changes I made and integrating them directly into the OS. I've been having problems with the SDIO interrupt most likely due to 2 copies of SD code.

jetcode · 2009-05-20 18:19:55

I just made life mega-easy. I took the changes required and integrated them directly into the OS and then tested my app using the OS filesystem and it works like a charm and provides multi-block read and write capability. I left the directory update unchanged so writes are updated for 1 or more sectors written depending on the length of the write.

Francis, I will send you GOOD source code.

Last edited by jetcode (2009-05-21 18:59:21)

Francis · 2009-05-21 07:03:20

Thanks Joe. We will test your new CircleOS, and we will publish it early next week.
The strategy is the following:
1. We push the developers registered on this forum to improve CircleOS.
2. In the future, we will set up a CVS server,
3. but, at the moment, the solution consists in sending us the modified files. We test them, and we publish the updates.
4. For the 'contest period', we don't add any new feature in CircleOS. However, we publish the required fixes.

jetcode · 2009-05-25 15:46:36

For everyone who uses the changes I added for multiple block read/write operations I need to state that the user MUST keep track of whether the multi-block write will fit in the current cluster. The multi-block write logic does not take this into consideration. It could but I thought of this after making the changes. For the most part if the user keeps the writes on cluster boundaries then everything will be fine. If a write requests 4 blocks but 2 blocks are available in the current cluster an error will occur because the new cluster is generated after the blocks are written.

<new>
Francis if you want I can fix this consideration by testing for the number blocks against the number of blocks left in the cluster and writing only whats left in the cluster up to the write length and then creating a new cluster and finishing the write by writing the remaining blocks. This approach requires no special handling. Give me a day or two and I'll see if I can put this logic in before the final release to CircleOS.

Last edited by jetcode (2009-05-25 19:05:07)

Socrates · 2009-05-26 14:54:13

Hello together,

i'm working on an application which need a filesystem on the SD-Card too. What i figured out until now, is there are some functions missing: DFS_CreateDir, DFS_CreateFile, DFS_RemakeDir and DFS_WriteFile above many sectors. Will there be solutions for my needs in the near future or do I have to make my own solutions?

P.S.: Many thanks to the community for the great work on primer and primer2!

jetcode · 2009-05-26 16:00:16

Files are created by opening with the write attribute. Usually there is an append attribute but that is missing from this implementation. Writing multiple sectors is coming in a future release. Directory support is minimal and you will have to roll your own and contribute to the project.

sfenwick76 · 2010-02-08 23:16:47

Hello,

Does anyone know if the speed enhancements that jetcode talked about in April 2009 were added to Circle OS? I am using 3.8.

Thanks,
Stephen

# 1 2009-03-08 15:45:33 SDIO+FAT speed

SDIO+FAT speed

# 2 2009-03-09 07:16:15 SDIO+FAT speed

Re: SDIO+FAT speed

# 3 2009-03-09 07:23:53 SDIO+FAT speed

Re: SDIO+FAT speed

# 4 2009-03-09 16:39:11 SDIO+FAT speed

Re: SDIO+FAT speed

# 5 2009-03-09 17:32:23 SDIO+FAT speed

Re: SDIO+FAT speed

# 6 2009-05-05 03:39:01 SDIO+FAT speed

Re: SDIO+FAT speed

# 7 2009-05-17 00:35:07 SDIO+FAT speed

Re: SDIO+FAT speed

# 8 2009-05-17 01:26:27 SDIO+FAT speed

Re: SDIO+FAT speed

# 9 2009-05-17 15:28:29 SDIO+FAT speed

Re: SDIO+FAT speed

# 10 2009-05-18 06:45:07 SDIO+FAT speed

Re: SDIO+FAT speed

# 11 2009-05-18 07:07:54 SDIO+FAT speed

Re: SDIO+FAT speed

# 12 2009-05-18 17:22:13 SDIO+FAT speed

Re: SDIO+FAT speed

# 13 2009-05-19 16:20:23 SDIO+FAT speed

Re: SDIO+FAT speed

# 14 2009-05-19 17:26:48 SDIO+FAT speed

Re: SDIO+FAT speed

# 15 2009-05-20 06:57:58 SDIO+FAT speed

Re: SDIO+FAT speed

# 16 2009-05-20 15:45:47 SDIO+FAT speed

Re: SDIO+FAT speed

# 17 2009-05-20 16:15:44 SDIO+FAT speed

Re: SDIO+FAT speed

# 18 2009-05-20 16:52:51 SDIO+FAT speed

Re: SDIO+FAT speed

# 19 2009-05-20 17:52:23 SDIO+FAT speed

Re: SDIO+FAT speed

# 20 2009-05-20 18:19:55 SDIO+FAT speed

Re: SDIO+FAT speed

# 21 2009-05-21 07:03:20 SDIO+FAT speed

Re: SDIO+FAT speed

# 22 2009-05-25 15:46:36 SDIO+FAT speed

Re: SDIO+FAT speed

# 23 2009-05-26 14:54:13 SDIO+FAT speed

Re: SDIO+FAT speed

# 24 2009-05-26 16:00:16 SDIO+FAT speed

Re: SDIO+FAT speed

# 25 2010-02-08 23:16:47 SDIO+FAT speed

Re: SDIO+FAT speed

Board footer