OSX considered harmfull

classic Classic list List threaded Threaded
12 messages Options
Wichert Akkerman Wichert Akkerman
Reply | Threaded
Open this post in threaded view
|

OSX considered harmfull

We're seeing more problems with releases made on OSX: those contain
._ files with the OSX resource fork information. Normally that is a
cosmetic problem only, but for translations it is worse: Zope will
abort with an error when it encounters those in
locales/<lanage>/LC_MESSAGES.

So please: double-check if your upload does not contain those before
uploading it.

Wichert.

--
Wichert Akkerman <[hidden email]>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Wichert Akkerman Wichert Akkerman
Reply | Threaded
Open this post in threaded view
|

Re: OSX considered harmfull

Previously Wichert Akkerman wrote:
> We're seeing more problems with releases made on OSX: those contain
> ._ files with the OSX resource fork information. Normally that is a
> cosmetic problem only, but for translations it is worse: Zope will
> abort with an error when it encounters those in
> locales/<lanage>/LC_MESSAGES.
>
> So please: double-check if your upload does not contain those before
> uploading it.

And for those of you who have upgraded to Leopard: apparently
COPY_EXTENDED_ATTRIBUTES_DISABLE no longer works - you have to use
COPYFILE_DISABLE now.

Wichert.

--
Wichert Akkerman <[hidden email]>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
grahamperrin grahamperrin
Reply | Threaded
Open this post in threaded view
|

Understanding ._ dot underscore and .DS_Store files

In reply to this post by Wichert Akkerman
._ files with the OSX resource fork information.
If a file on NTFS or ZFS or HFS+ has a resource fork (a rarity), you can strip that fork before copying, but even if you:

* copy an unforked file
* using Finder
* using a protocol, or to a filing system, that lacks support

then you will probably find, to your surprise, a
._ dot underscore file at the destination

-- at least, for as long as the file is being copied.

For a more detailed explanation, see
<http://www.plone4artists.org/products/plone4artistsaudio/issues/32>.

These are not bugs in Finder or Mac OS X. More often, people simply misunderstand the richness of filing systems such as HFS+ ... Finder is simply doing its job, ensuring that data/metadata is not lost.

When I last considered alternatives/complements to Finder, muCommander was most interesting.
Mentioned at <http://groups.google.com/group/MacFusion-devel/browse_frm/thread/8af1d90b40c4d97c/>.

FWIW in a broader WebDAV context I hope that
wdfs <http://noedler.de/projekte/wdfs/> may be preferable to
/System/Library/Filesystems/webdav.fs/
in Mac OS X 10.4. If you're interested in this aspect, see
<http://groups.google.com/group/MacFusion-devel/browse_frm/thread/2089d41bae8fb1a8> and
<http://groups.google.com/group/macfuse-devel/browse_frm/thread/26a891bb3143701c>.

Best regards

Graham
MacFusion project
Ricardo Newbery-2 Ricardo Newbery-2
Reply | Threaded
Open this post in threaded view
|

Re: Understanding ._ dot underscore and .DS_Store files


On Nov 14, 2007, at 8:38 PM, Graham Perrin wrote:

>
>
>
>> ._ files with the OSX resource fork information.
>>
>
> If a file on NTFS or ZFS or HFS+ has a resource fork (a rarity),  
> you can
> strip that fork before copying, but even if you:
>
> * copy an unforked file
> * using Finder
> * using a protocol, or to a filing system, that lacks support
>
> then you will probably find, to your surprise, a
> ._ dot underscore file at the destination
>
> -- at least, for as long as the file is being copied.
> For a more detailed explanation, see
> <http://www.plone4artists.org/products/plone4artistsaudio/issues/32>.
>
> These are not bugs in Finder or Mac OS X. More often, people simply
> misunderstand the richness of filing systems such as HFS+ ...  
> Finder is
> simply doing its job, ensuring that data/metadata is not lost.


I'm not sure but I believe the issue Wichert is referring to is *not*  
the behavior we have to deal with when copying files from an HFS+  
file system.  He's not talking about file transfer but rather the  
packaging up of releases.

The issue is with the use of the 'tar' function on the OS X command  
line.  From what I understand, the tar function is creating these ._*  
files in order to preserve the resource forks.  If this is not what  
you want, one solution is to add the following line to your .profile  
file:

export COPY_EXTENDED_ATTRIBUTES_DISABLED=true

or if you're on Leopard:

export COPYFILE_DISABLE=true

Ric






-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Wichert Akkerman Wichert Akkerman
Reply | Threaded
Open this post in threaded view
|

Re: Understanding ._ dot underscore and .DS_Store files

In reply to this post by grahamperrin
Previously Graham Perrin wrote:

>
>
>
> > ._ files with the OSX resource fork information.
> >
>
> If a file on NTFS or ZFS or HFS+ has a resource fork (a rarity), you can
> strip that fork before copying, but even if you:
>
> * copy an unforked file
> * using Finder
> * using a protocol, or to a filing system, that lacks support
>
> then you will probably find, to your surprise, a
> ._ dot underscore file at the destination
>
> -- at least, for as long as the file is being copied.
>
> For a more detailed explanation, see
> <http://www.plone4artists.org/products/plone4artistsaudio/issues/32>.
>
> These are not bugs in Finder or Mac OS X. More often, people simply
> misunderstand the richness of filing systems such as HFS+ ... Finder is
> simply doing its job, ensuring that data/metadata is not lost.

I understand the richness of the filesystem very well. What I don't
understand is why OSX tools prefer to create lots of special files
instead of using standard options to hide them. For example the tar
format is quite flexible and allows you to add custom block types. If
Apple had used that this problem would not exist.

Wichert.

--
Wichert Akkerman <[hidden email]>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Ricardo Newbery-2 Ricardo Newbery-2
Reply | Threaded
Open this post in threaded view
|

Re: Understanding ._ dot underscore and .DS_Store files


On Nov 14, 2007, at 11:32 PM, Wichert Akkerman wrote:

> I understand the richness of the filesystem very well. What I don't
> understand is why OSX tools prefer to create lots of special files
> instead of using standard options to hide them. For example the tar
> format is quite flexible and allows you to add custom block types. If
> Apple had used that this problem would not exist.
>
> Wichert.


It's my understanding that the metadata support in tar that was added  
in 10.4 is still experimental, but I believe the intent was to plug  
into the same API that the other OS X tools use to keep track of this  
metadata.  It probably seemed natural to represent the metatadata  
using the AppleDouble format, which after all has been around for  
awhile and is supported by a host of other tools.

Perhaps, as you say, it might have been useful to add a "custom block  
type" to handle this.  But not knowing the technical issues involved,  
I'm hesitant to criticize these sorts of decisions.  I understand  
that the regular tar utility doesn't even *see* the metadata because  
it doesn't know how to get this information from the HFS+ filesystem  
API.  So they need to add this ability to tar.  I suspect that in  
order to get two-way translation of this metadata, you probably need  
a sane fallback in case someone uses a non-apple tar on a non-HFS+  
filesystem, so the AppleDouble format with hidden files may make a  
lot of sense.  Could you even get this sort of sane fallback using  
custom block types?

What I think they fumbled on is not providing some sort of command  
line option to turn off the retention of metadata when using tar.  
Using an environment variable to turn this off seems a little  
clumsy.  I noticed that the Mac version of rsync has an "extended-
attributes" option "-E" that turns on support for metadata.

Ric



-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
grahamperrin grahamperrin
Reply | Threaded
Open this post in threaded view
|

Mac OS X 10.5 ._ and .DS_Store files: cp -XR (before tar) to lose Extended Attributes (EAs) and resource forks

Ricardo Newbery wrote
What I think they fumbled on is not providing some sort of command line option to turn off the retention of metadata when using tar. Using an environment variable to turn this off seems a little
clumsy.  
The variables are definitely useful (thanks wichert) but I might hesitate before adding to my
~/.profile

I noticed that the Mac version of rsync has an "extended-attributes" option "-E" that turns on support for metadata.
Similarly in Mac OS X 10.5, for the cp command
<http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/cp.1.html>

-X    Do not copy Extended Attributes (EAs) or resource forks.

So users of 10.5 who are less keen on
COPYFILE_DISABLE
might instead
cp -XR
then tar the result.

-X is not an option for cp in 10.4, unfortunately.
grahamperrin grahamperrin
Reply | Threaded
Open this post in threaded view
|

the ditto command approach to ._ and .DS_Store files

In reply to this post by Wichert Akkerman
For archiving with special attention to resource forks and extended attributes, there's also the lesser-known
ditto
but I don't know whether CPIO or PKZip archiving are acceptable alternatives to tar.

From my Mac OS X 10.4.10 man for ditto:

     --norsrc      do not preserve resource forks and HFS meta-data.  If both
                   --norsrc and --rsrc are passed, whichever is passed last
                   will take precedence.  Both options override DITTONORSRC.

man for the more recent version of ditto is at
<http://developer.apple.com/DOCUMENTATION/Darwin/Reference/ManPages/man1/ditto.1.html> and here's another option

--noextattr
grahamperrin grahamperrin
Reply | Threaded
Open this post in threaded view
|

for tar excluding ._ try muCommander

In reply to this post by Wichert Akkerman
wichert wrote
Zope will abort with an error when it encounters [ ._ files] in
locales/<lanage>/LC_MESSAGES.
If
._
is problematic but
.DS_Store
is not problematic, then muCommander
<http://www.mucommander.com/> may be useful.

muCommander is apparently blind to resource forks,
<http://www.mucommander.com/forums/viewtopic.php?t=509>.

A brief test this morning showed me that .tar created by muCommander:

* includes .DS_Store files
* excludes ._ files.  

I'll quieten down now, that's more than enough!
Wichert Akkerman Wichert Akkerman
Reply | Threaded
Open this post in threaded view
|

Re: for tar excluding ._ try muCommander

Previously Graham Perrin wrote:
> If
> ._
> is problematic but
> .DS_Store
> is not problematic, then muCommander
> <http://www.mucommander.com/> may be useful.

.DS_Store is just as bad.

Wichert.

--
Wichert Akkerman <[hidden email]>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Steve McMahon Steve McMahon
Reply | Threaded
Open this post in threaded view
|

Re: Mac OS X 10.5 ._ and .DS_Store files: cp -XR (before tar) to lose Extended Attributes (EAs) and resource forks

In reply to this post by grahamperrin
 > So users of 10.5 who are less keen on
 > COPYFILE_DISABLE
 > might instead
 > cp -XR
 > then tar the result.

I'm pretty sure that Leopard's tar is generating and adding the ._*
files itself. So, this is probably not a solution.

Graham Perrin wrote:

>
> Ricardo Newbery wrote:
>> What I think they fumbled on is not providing some sort of command line
>> option to turn off the retention of metadata when using tar. Using an
>> environment variable to turn this off seems a little
>> clumsy.  
>>
>
> The variables are definitely useful (thanks wichert) but I might hesitate
> before adding to my
> ~/.profile
>
>
>
>> I noticed that the Mac version of rsync has an "extended-attributes"
>> option "-E" that turns on support for metadata.
>>
>
> Similarly in Mac OS X 10.5, for the cp command
> <http://developer.apple.com/documentation/Darwin/Reference/ManPages/man1/cp.1.html>
>
> -X    Do not copy Extended Attributes (EAs) or resource forks.
>
> So users of 10.5 who are less keen on
> COPYFILE_DISABLE
> might instead
> cp -XR
> then tar the result.
>
> -X is not an option for cp in 10.4, unfortunately.
>

--
______________________________________________________

Steve McMahon
Reid-McMahon, LLC
[hidden email]
[hidden email]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Marco De Vitis Marco De Vitis
Reply | Threaded
Open this post in threaded view
|

Re: for tar excluding ._ try muCommander

In reply to this post by grahamperrin
Il 15/11/2007 15:02, Graham Perrin ha scritto:

> If
> ._
> is problematic but
> .DS_Store
> is not problematic, then muCommander
> <http://www.mucommander.com/> may be useful.

Sorry for being so late, but... if this thread was meant for developers
going to pack and upload their products, and developers are ok with
using a GUI app like muCommander for packing, then why not CleanArchiver?
http://www.sopht.jp/cleanarchiver/
It can exclude all OSX special files from the archives it creates.

--
Ciao,
   Marco.


-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers