! Please note that this is a snapshot of our old Bugzilla server, which is read only since May 29, 2020. Please go to gitlab.xfce.org for our new server !
Thunar SendTo Email: improve archives detection
Status:
RESOLVED: FIXED

Comments

Description Sergey Ponomarev 2019-09-02 15:08:27 CEST
Created attachment 8983 
sendto plugin: get rid of archives content types and use  shared-mime-info instead

When user selects few files to send then internally sendto-email checks that all files are already archives and if not it shows a dialog window to ask to compress the files (see tse_ask_compress function).
To detect that the file is an archive the plugin contains a full list of archive MIME types on plugins/thunar-sendto-email/main.c:78:

/* well known archive types */
static const char *TSE_MIME_TYPES[] = {
  "application/x-ar",
   .. 
  "application/x-tar",
  "application/zip"
}

The problem here is that in fact there are more compressor algorithms. In the TSE_MIME_TYPES list is missing LZMA and recently was published ZStandard (zstd) *.zst and Brotli *.br.

I wanted to add them but in fact I decided to find a way to get rid of the TSE_MIME_TYPES list at all.
To determine a content type internally the sendto-email calls the  g_file_info_get_content_type function from GLib which uses the Shared MIME Info library https://www.freedesktop.org/wiki/Software/shared-mime-info/ which is kind of clone of libmagic + icons + translations.

Unfortunately the shared-mime-info doesn't have a clear flag that the type is an archive. But all archive types (tar, zip, gz etc) always have the same icon "package-x-generic".
So we can use the content type's icon name to determine that it is an archive or compressed file:

  generic_icon_name = g_content_type_get_generic_icon_name (content_type);
  file_is_archive = g_strcmp0 (generic_icon_name, "package-x-generic") == 0;

This looks like a hack but it always works and when zst and br or any other will be added to Shared MIME Info then the plugin starts to recognize them.

I implemented this solution but before I did a small refactoring and bug fix:
#15916 BUG: Thunar SendTo Email doesn't detects files that are already archives
https://bugzilla.xfce.org/show_bug.cgi?id=15916

So my patch assumes that the patches from that bug was already accepted
Comment 1 alexxcons editbugs 2019-09-03 23:14:09 CEST
Thanks for the patch !

Hmm, I am not yet full convinced on this one ... using an icon name to check if something is an archive indeed sounds like a hack.

> Unfortunately the shared-mime-info doesn't have a clear flag that the type is an archive.
> But all archive types (tar, zip, gz etc) always have the same icon "package-x-generic".
Is it as well the other way around ... are all files with "package-x-generic" archives ?

I would at least open a bug against https://gitlab.gnome.org/GNOME/glib in order to forsee a proper way to check "is archive" in the future.
Comment 2 alexxcons editbugs 2019-09-03 23:39:23 CEST
I just aked another xfce4-dev to have a second opinion on that .... sorry, it looks like you will need to just add to missing mime types.

But let me say that you prooved creativity. Your approach is at least interesting !  :)
Comment 3 Sergey Ponomarev 2019-09-04 09:28:38 CEST
> are all files with "package-x-generic" archives ?
yes, I checked in https://github.com/freedesktop/xdg-shared-mime-info/blob/master/freedesktop.org.xml.in


> would at least open a bug against glib

yes, I'll try to do this latter but here is a problem that I'm not so experienced to send such PR into GLib itself. I'm just a regular Java dev and for me it's much harder. Another problem is a time: 
1. we need to patch the shared-mime-info lib (it may take some time to prepare a patch)
2. then wait until all distros will update it (API changed i.e. major update: I guess this may take few months, maybe even year)
3. then we need to patch GLib
4. then wait until all distros will update it (I'm afraid this may take years)
5. only then update XFCE

This may takes years while my main goal is to force wider adoption of ZStandard and Brotli.

In the same time this approach even given that looks hacky but it's simple and well documented.
So when the zstd and brotli will be added to shared-mime-info (just adding the two types should much more easy to add) they begin to be recognized almost immediately.

Also please note that this patch is in fact just a simplified form of main change
https://bugzilla.xfce.org/show_bug.cgi?id=15918

And there is a main goal to speedup archives detection and to become prepared for zstd and brotli archives.


> just add to missing mime types.
yes, that what I did first and if you decide not to accept this detection improvement then at least add these missing types:


Index: plugins/thunar-sendto-email/main.c
===================================================================
--- plugins/thunar-sendto-email/main.c	(date 1567420848000)
+++ plugins/thunar-sendto-email/main.c	(date 1567582010442)
@@ -76,8 +76,12 @@
 
 /* well known archive types */
 static const char *TSE_MIME_TYPES[] = {
+  "application/x-7z-compressed",
+  "application/x-7z-compressed-tar",
   "application/x-ar",
   "application/x-arj",
+  "application/x-brotli",
+  "application/x-brotli-compressed-tar",
   "application/x-bzip",
   "application/x-bzip-compressed-tar",
   "application/x-compress",
@@ -87,9 +91,13 @@
   "application/x-gzip",
   "application/x-lha",
   "application/x-lhz",
+  "application/x-lzma",
+  "application/x-lzma-compressed-tar",
   "application/x-rar",
   "application/x-rar-compressed",
   "application/x-tar",
+  "application/x-xz",
+  "application/x-xz-compressed-tar",
   "application/x-zip",
   "application/x-zip-compressed",
   "application/zip",
@@ -100,6 +108,8 @@
   "application/x-lzop",
   "application/x-zoo",
   "application/x-cd-image",
+  "application/zstd",
+  "application/x-zstd-compressed-tar",
 };
Comment 4 alexxcons editbugs 2019-09-04 23:22:56 CEST
(In reply to Sergey Ponomarev from comment #3)
> > are all files with "package-x-generic" archives ?
> yes, I checked in
> https://github.com/freedesktop/xdg-shared-mime-info/blob/master/freedesktop.
> org.xml.in
> 
> 
> > would at least open a bug against glib
> 
> yes, I'll try to do this latter but here is a problem that I'm not so
> experienced to send such PR into GLib itself. I'm just a regular Java dev
> and for me it's much harder.
You dont need to send a PR. Just open a bug should be fine.

> Another problem is a time: ....
I did not want to wait for the next glib. We can fix it in a different way now, and use the proper glib method whenever there is one available.

> Also please note that this patch is in fact just a simplified form of main
> change
> https://bugzilla.xfce.org/show_bug.cgi?id=15918
Ok ... did not see that one (added rss feed for t-a-p now)

Well, so why we need to have a separate "isArchive" check here, when we can re-use your thunarx API ?
Comment 5 Sergey Ponomarev 2019-09-05 06:06:07 CEST
> Just open a bug should be fine.

Great, I'll leave a link here when I do this.

> why we need to have a separate "isArchive" check here, when we can re-use your thunarx API ?

That's a good question. I also wanted to do like this but it turned out that the sendto-email plugin is not a shared library but a separate command line tool /usr/local/lib/Thunar/thunar-sendto-email so it doesn't use the thunarx API at all.

> We can fix it in a different way now

If the hack is small, simple, well documented, localized into a function which internal implementation can be easily changed latter, backward compatible and in the same time more flexible then I'm pretty sure it's ok to use it.
Comment 6 Sergey Ponomarev 2019-09-05 08:12:47 CEST
BTW consider this: the listing of all archive MIME types is also a hack, and very dirty: we already have a problem when user sees an archive icon on the file but can't extract it from the menu. Yes, potentially there can be a situation when the archive type is not supported by the archive manager e.g. XArchiver doesn't support Brotli yet, but this will be eventually fixed.
Comment 7 alexxcons editbugs 2019-09-06 13:27:08 CEST
> Great, I'll leave a link here when I do this.
Thanks !

> That's a good question. I also wanted to do like this but it turned out that the sendto-email plugin is not a shared library but a separate 
> command line tool /usr/local/lib/Thunar/thunar-sendto-email so it doesn't use the thunarx API at all.
Meh, that's a pity .. ok, I'll apply the fix later on.

> If the hack is small, simple, well documented, localized into a function which internal implementation can be easily changed latter,
> backward compatible and in the same time more flexible then I'm pretty sure it's ok to use it.
And still it is a hack. I dont think much is gained by replacing one hack by another.

> BTW consider this: the listing of all archive MIME types is also a hack, and very dirty: we already have a problem when user sees an archive icon on the file but can't extract it from the menu. Yes, potentially there can be a situation when the archive type is not supported by the archive manager e.g. XArchiver doesn't support Brotli yet, but this will be eventually fixed.
As you said yourself. Using the other hack may bring other pitfalls .. e.g. attempt to load unsupported archive formats. As well the icon name might change in the future. How about a different icon for uncompressed archives ?
Comment 8 Sergey Ponomarev 2019-09-06 21:10:37 CEST
> As well the icon name might change in the future

I think this is a change of the same level as MIME content type changed. Even if this happens hope that will be an official way to determine archive types.
Comment 9 Sergey Ponomarev 2019-09-06 22:52:43 CEST
I created the issue ticket for 
xdgmime  https://gitlab.freedesktop.org/xdg/xdgmime/issues/29 Get generic type of MIME (i.e. "image" from "image/png")
Please check it to  add your thoughts/votes or just subscribe.
Comment 10 Git Bot editbugs 2019-09-06 23:20:59 CEST
Sergey Ponomarev referenced this bugreport in commit 8583ac0b5de00c8bf5c2028d272f0e3d3838cb91

Thunar SendTo Email: Add missing archive formats for archive detection (Bug #15917)

https://git.xfce.org/xfce/thunar/commit?id=8583ac0b5de00c8bf5c2028d272f0e3d3838cb91
Comment 11 Git Bot editbugs 2019-09-06 23:22:16 CEST
Sergey Ponomarev referenced this bugreport in commit fe65406cdaf750ceec1ffac1d5c0b29a79596b5c

Thunar SendTo Email: Add missing archive formats for archive detection (Bug #15917)

https://git.xfce.org/xfce/thunar/commit?id=fe65406cdaf750ceec1ffac1d5c0b29a79596b5c
Comment 12 alexxcons editbugs 2019-09-06 23:37:24 CEST
Pushed the simple fix to master and 4.14 now. Thanks for your effort !

(In reply to Sergey Ponomarev from comment #9)
> I created the issue ticket for 
> xdgmime  https://gitlab.freedesktop.org/xdg/xdgmime/issues/29 Get generic
> type of MIME (i.e. "image" from "image/png")
> Please check it to  add your thoughts/votes or just subscribe.

Thanks ! Wonder if xdgmime is the correct project for it .. hopefully the maintainer will tell us.

Bug #15917

Reported by:
Sergey Ponomarev
Reported on: 2019-09-02
Last modified on: 2019-09-06

People

Assignee:
Xfce Bug Triage
CC List:
1 user

Version

Attachments

Additional information