v0.9.7. Windows 10 can't see changes on the device.

[trinhnhatnam]

This issue was reported by UHU-ger previously but was closed.

Steps to reproduce:

• Make sure Windows 10 can see the content of a folder on the device running umtprd.
• On the device, create a new file called "tmp.txt".
• "tmp.txt" doesn't show up on Windows 10. Refreshes don't help. Leaving/re-entering the folder doesn't help.
• Unplug/Replug USB. Windows 10 can now see "tmp.txt".

With a Linux host, Linux Mint 19 Tara XFCE specifically, "tmp.txt" shows up after a refresh.

By the way, in "umtprd-ffs.sh", "mtp" should be replaced by "umtp" in line "mount -t functionfs mtp /dev/ffs-umtp". At least, I had to do that.

[jfdelnero]

Which kernel version are you using ?
is the inotify feature enabled ? Can you post here your umtp config file ?

Can you rebuild umtprd in debug mode, test some files addition/removal and put here the logs ?

[trinhnhatnam]

My set up:

• Custom board based on sama5d2Xplained.
• Based on linux-linux4sam_5.8 from https://github.com/linux4sam/linux-at91. Kernel is based on 4.9.87, with RT_PREEMPT patch.

Config file and startup script are below. I'll add debug log later. Thanks!

Config:
------------------------
# Loop / daemon mode
# Set to 1 to don't shutdown uMTPrd when the link is disconnected.
loop_on_disconnect 1

#storage command : Create add a storage entry point. Up to 16 entry points supported
#Syntax : storage "PATH" "NAME"
storage "/"      "root folder"
storage "/home"  "home folder"

# Set the USB manufacturer string
manufacturer "Viveris Technologies"

# Set the USB Product string
product "The Viveris Product !"

# Set the USB Serial number string
serial "01234567"

# Set the USB interface string. Should be always "MTP"
interface "MTP"

# Set the USB Vendor ID, Product ID and class
usb_vendor_id  0x1D6B # Linux Foundation
usb_product_id 0x0100 # PTP Gadget
usb_class 0x6         # Image
usb_subclass 0x1      # Still Imaging device
usb_protocol 0x1      #

# Device version
usb_dev_version 0x3008

usb_functionfs_mode 0x1
usb_dev_path   "/dev/ffs-umtp/ep0"
usb_epin_path  "/dev/ffs-umtp/ep1"
usb_epout_path "/dev/ffs-umtp/ep2"
usb_epint_path "/dev/ffs-umtp/ep3"

# Max USB packet size
usb_max_packet_size 0x200
------------------------------------


Startup script:
--------------------------------------------
	modprobe libcomposite
	mount -t configfs none /sys/kernel/config
	cd /sys/kernel/config/
	cd usb_gadget
	mkdir g1
	cd g1
	echo "0xEF" > bDeviceClass
	echo "0x02" > bDeviceSubClass
	echo "0x01" > bDeviceProtocol
	echo "0x0010" > bcdDevice
	echo "0x04D8" > idVendor
	echo "0x1234" > idProduct
	mkdir strings/0x409
	echo "0123456789" > strings/0x409/serialnumber
	echo "ABC" > strings/0x409/manufacturer
	echo "XYZ" > strings/0x409/product
	mkdir functions/ffs.umtp
	mkdir functions/eem.usb0
	mkdir configs/c.1
	mkdir configs/c.1/strings/0x409
	echo "MTP+EEM" > configs/c.1/strings/0x409/configuration
	ln -s functions/ffs.umtp configs/c.1
	ln -s functions/eem.usb0 configs/c.1
	mkdir /dev/ffs-umtp
	mount -t functionfs umtp /dev/ffs-umtp
	# Start the umtprd service
	umtprd &
	sleep 1
	echo `ls /sys/class/udc` > UDC
-----------------------------------------------------------------------------

[jfdelnero]

I see that you have changed the vendor & product id in the startup script. Can you retry with the original Linux Foundation/PTP gadget ids ?

[trinhnhatnam]

I did and it didn't help.

But I found out a couple of things. My config has "/home" but the device doesn't have it. After removing "/home" from config, I don't see the reported problem anymore. That's good news. And with the change, Linux host now auto-updates (before, I had to refresh).

I see a different problem now. I remove "tmp.txt" and refresh Windows 10. The refresh takes a long time. After that, Windows 10 doesn't see updates anymore. Refreshes don't help. The problem doesn't exist with my Linux host. I can open a new "issue" if you want.

[jfdelnero]

Ok then i guess that the kernel inotify need a valid entry point. i will add the some sanity checks to address this possible issue.

Regarding Windows 10 i will do some more tests but i have reproduced lots of issues with an Android 6 smartphone. It looks like there are some regressions in the Windows 10 initiator versus the Windows 7 initiator.

[trinhnhatnam]

I just came back and tried v0.9.7 again, and I couldn't see the problem mentioned in my previous post.
I guess updates to my Windows 10 since Jan 24 have fixed the problem.

[UHU-ger]

It seems I got problems with Win 10 and refreshing again/too. (Using v1.0.0 and FunctionFS Mode on Beaglebone Black.) I use 2 Folders. In the first attempt, one folder (with option read-only in conf) showed new Files (i mean created on BeBoBlack) when refreshing. The other Folder (in which i had copied a File from Win10 before) did not show new Files when refreshing.
Then i disconnected Usb and reconnected.
The Files created before are now shown, but now refreshing does not do anything on both folders.
I will try to make further tests and compare with the behavior on Linux and report again with more details if i find anything.

[jfdelnero]

Are you using a regular folder to test the refresh ? the inotify scheme currently used to detect and report the changes doesn't work well with mounted network disk for example. Please also note that the disk paths specified in the config file must be available at startup otherwise the inotify layer will not work properly.

[jfdelnero]

Ok issue reproduced : The problem is that with the current implementation an entry is registered one time for each drive. The inotify api doesn't support this (EDIT : It does support this). I am working on a fix.

[jfdelnero]

The issue is now fixed on the trunk.

[UHU-ger]

I assume with trunk you mean the master branch? If not: how do i get the trunk branch?

TLDR: automatic refresh seems to refresh a wrong folder (maybe counting error starting with 0 & elsewhere with 1 ?)

Long description + Makefile for automated installation on BeagleBoneBlack
BeagleBoneBlack-makefile.tar.gz
Done some testing, found Problems even with Linux (Mint 19.4) (did not try Win10 at the moment, because i found errors already in Linux)
I use 2 empty Folders, and did not remove the config to the /www folder.
When i create a file in one of the folders (from host with: touch 1.txt) sometimes actualisation works, but mostly not. I found in the debug output that it does not check the folder that is opened from Client (in this case i was in the one called incoming), but it looks in /www

Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] Entry 0 - /
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] mtp_get_storage_root : FFFF0003
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] mtp_get_storage_root : FFFF0003 -> /www
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] build_full_path : / -> /www//
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] add_entry : File not present - add entry ()
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] mtp_push_event : Event packet buffer - 16 Bytes :
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] 10 00 00 00 04 00 02 40 01 00 00 00 01 00 00 00 .......@........
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] write_usb return: 16
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] inotify_thread (IN_CREATE): Entry 1.txt created (Handle 0x00000001)
Jan 20 10:37:33 beaglebone umtprd: [uMTPrd - Debug] inotify_thread (IN_CREATE): Watch point descriptor not found in the db ! (Descriptor 0x00000001)
`
Then i stopped umtprd with make kill, commented the /www directory in /etc/umtprd/umtprd.conf
and restarted with make start.

Now the folder incoming does not refresh itself

And a view in my second folder (from client) shows an file which is shown as "null" with 4,1kB in the file explorer from mint. After i made touch 3.txt from beagle it did not show up, but i noticed that if i refresh the folder with F5 "null" File disapears and 3.txt is shown. But if i leave the folder output and then go back in null is back again (dissapearing when gitting F5 again)
If i open the directory in the shell and type ls it says ls: reading directory '.' Input/output error

I also tryed without config to /www and i saw different errors,

Errors i have seen earlier:
add_entry : File not present - add entry () <-- seen with the File in the Brackets and without (then automatic refresh die not work iirc)

also, propably the same Bug: a little later:
build_full_path : -> /umtprd-inout/outgoing/

I just noticed this is propably the same bug as with /www, because i created a file with the beagle in incoming and it reads in outgoing.

My setup:
I attached the Makefile i use, together with the patches for the config files i use, so you can easily see my settings.
I take a fresh Beagle (with the 9.5 IOT Image)
I untar the compressed file to ~/umtp-files and then:
make beagleboneblack (this deactivates the usb functions the bbb is delivered with (including restart), will propably not work correctly on other Boards)
make install-debug
make start
Now "make start" starts umtprd (i added some lines to print the pid in a file, so it can be killed more easily with "make kill") and opens the syslog (if install-debug was chosen)

[jfdelnero]

Thanks for the detailed informations, i will try to reproduce the issue.

I assume with trunk you mean the master branch? If not: how do i get the trunk branch?

Yes the master branch (yes github use git, not svn ;-) )

[UHU-ger]

Further Informations:
If i use only 1 Folder it seem to work a lot better.

But while testing around i found following bug (and i guess it may be connected to this bug):
If i copy a file (on beagle) into the directory which i use for mtp, the file is shown on client. But its file size shows 0 and if i try opening the file it is empty. After unpluggin and reattatching the usb the files are shown/working correct.
same on Linux mint & Win 10 (win needs F5 to refresh, mint does automatic)

[devangpatel]

I have the same issue with WIN 10 does not show the content of the file. My embedded Linux does export some CSV file while that device is connected to WIN 10 machine. Once I hit F5 or refresh the Windows explorer then File does show up with file size as 0. If I do then disconnect and connect the device again with PC then the same file show up proper size and content inside. Event latest master branch code does solve my problem.

[jfdelnero]

@devangpatel @UHU-ger the "ObjectInfoChanged" event is now implemented and the file size is now updated. Tested and working on Windows 7. Linux mint doesn't appear to care about this event...

[UHU-ger]

Works better, but i found a little more:

I made a empty File and add 3 lines with: something

debian@bb1:/umtprd-inout$ touch t3.txt
debian@bb1:/umtprd-inout$ echo something >> t3.txt
debian@bb1:/umtprd-inout$ echo something >> t3.txt
debian@bb1:/umtprd-inout$ echo something >> t3.txt

The behavior i found on Win 10:
The file size changes when hitting F5, but size and data are 1 echo behind.

Means: after 3x echo and then refresh on Windows, the file size and the data shown are on the level from 2x echo.
every following echo to add a line behaves the same, the last change does not show, but the next-to-last change, which did not show before shows up.

Got only a ubuntu19.04 as VM on Win 10 to test with Linux here today, and because i am not sure that the virtualisation does not add Errors i did not try on Linux today.

regards

[jfdelnero]

I suspect that some bugs is related to the windows mtp initiator/explorer... What i miss is a device with another mtp responder to compare the behaviour.

[devangpatel]

I have another battle to deal with. Now with latest Master COMMIT build, my WIN 10 does enumerate with my embedded device and I could see my product name under Win 10 Devices under Settings. However File Explorer does not even show mounted device.

We have been using "78d244b" this version of COMMIT ID that works with Win 10. (We still have Win 10 SYNC issue as stated in above comments. We have ignored this issue for now.)

[jfdelnero]

I have just tested the latest master commit on a Windows 10 without any issue. Which mode are you using ? gadgetfs or functionsfs ? Is the issue specific to this very last commit ? Can you open another issue and share the umtp configuration and startup files.

[devangpatel]

We use functionfs. Sure I will open another issue and will share those files. I want to do more debugging on my side first before I open an issue. Thank you very much for prompt response and help.

[jfdelnero]

We use functionfs. Sure I will open another issue and will share those files. I want to do more debugging on my side first before I open an issue. Thank you very much for prompt response and help.

Can you tell me if your issue is related to the last commit ? (I want to know if one of the recent commits are causing this issue or not).