Configure Plone to use Relstorage as blobstorage

classic Classic list List threaded Threaded
2 messages Options
Nicolas Göddel Nicolas Göddel
Reply | Threaded
Open this post in threaded view
|

Configure Plone to use Relstorage as blobstorage

Hi,

I have a installation of Plone 4.3.3 with one site. First the buildout was
configured to use the Data.fs file in var/filestorage and a shared blob storage
in var/blobstorage. Then I added a Relstorage to the buildout and converted the
content of the Data.fs file to the underlying MySQL database. Now Plone is using
Relstorage instead of Data.fs.

But now I want also to use the Relstorage for the blobs. Because I am relatively
new to Plone and especially the Relstorage thing, my idea was to first setup a
new empty Plone. Then I copied the buildout.cfg and base.cfg from the first one
to the new one. Then I created a new database userZodb and changed the base.cfg
for using the new database and I also changed the ports for zeoserver and
clients. The next step was to reconfigure the relstorage for not using the file
based blobstorage.

rel-storage =
    type mysql
    db userZodb
    user zodbuser
    passwd innzop
    blob-dir ${buildout:var-dir}/blobstorage
    shared-blob-dir false

# shared blobs are much faster if we're on the same server.
# if not, turn it off.
shared-blob = off

Then I ran the buildout. All was built successfully. After starting the
zeoserver, I got this error from the client:

user@server:~/Plone433-dev/zeocluster3$ ./bin/zeoserver start
.
daemon process started, pid=35136
user@server:~/Plone433-dev/zeocluster3$ ./bin/client1 fg
2014-12-17 14:50:31 INFO ZServer HTTP server started at Wed Dec 17 14:50:31 2014
    Hostname: 0.0.0.0
    Port: 9180
2014-12-17 14:50:32 INFO Products.PloneFormGen gpg_subprocess initialized, using
/usr/bin/gpg
Traceback (most recent call last):
  File "/home/user/Plone433-dev/zeocluster3/parts/client1/bin/interpreter", line
289, in <module>
    exec(compile(__file__f.read(), __file__, "exec"))
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/run.py",
line 76, in <module>
    run()
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/run.py",
line 22, in run
    starter.prepare()
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/__init__.py",
line 86, in prepare
    self.startZope()
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/__init__.py",
line 262, in startZope
    Zope2.startup()
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/__init__.py",
line 47, in startup
    _startup()
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/App/startup.py",
line 81, in startup
    DB = dbtab.getDatabase('/', is_root=1)
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py",
line 287, in getDatabase
    db = factory.open(name, self.databases)
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py",
line 185, in open
    DB = self.createDB(database_name, databases)
  File
"/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py",
line 182, in createDB
    return ZODBDatabase.open(self, databases)
  File
"/home/user/Plone433-dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/config.py",
line 101, in open
    storage = section.storage.open()
  File
"/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/config.py",
line 33, in open
    return RelStorage(adapter, name=config.name, options=options)
  File
"/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/storage.py",
line 212, in __init__
    self.blobhelper = BlobHelper(options=options, adapter=adapter)
  File
"/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/blobhelper.py",
line 118, in __init__
    fshelper.create()
  File
"/home/user/Plone433-dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/blob.py",
line 359, in create
    (self.layout_name, self.base_dir, layout))
ValueError: Directory layout `zeocache` selected for blob directory
/home/user/Plone433-dev/zeocluster3/var/blobstorage/, but marker found for
layout `bushy`

Unfortunality I have no idea where the problem could be.

I think the best would be to simply disable blobstorage completely. This way
Plone would be forced to use filestorage respectively the Relstorage for the
blobs. So I decided to comment out all lines defining "blob-storage =
${:var}/blobstorage". I thought Plone would now using Relstorage for the blobs
too. But after the buildout there is still the following line in
parts/zeoserver/etc/zeo.conf:

blob-dir /home/openmind/Plone433-dev/zeocluster3/var/blobstorage

Why want I do this? Because our customer wants only one big MySQL database which
stores filestorage and blobstorage. So he can backup his system easier. I know
this is not a good idea, but he wants so.

Any suggestions? Thank you!
_______________________________________________
Setup mailing list
[hidden email]
https://lists.plone.org/mailman/listinfo/plone-setup
Maurits van Rees-3 Maurits van Rees-3
Reply | Threaded
Open this post in threaded view
|

Re: Configure Plone to use Relstorage as blobstorage

Nicolas Göddel schreef op 18-12-14 12:28:

> Hi,
>
> I have a installation of Plone 4.3.3 with one site. First the buildout was
> configured to use the Data.fs file in var/filestorage and a shared blob storage
> in var/blobstorage. Then I added a Relstorage to the buildout and converted the
> content of the Data.fs file to the underlying MySQL database. Now Plone is using
> Relstorage instead of Data.fs.
>
> But now I want also to use the Relstorage for the blobs. Because I am relatively
> new to Plone and especially the Relstorage thing, my idea was to first setup a
> new empty Plone. Then I copied the buildout.cfg and base.cfg from the first one
> to the new one. Then I created a new database userZodb and changed the base.cfg
> for using the new database and I also changed the ports for zeoserver and
> clients. The next step was to reconfigure the relstorage for not using the file
> based blobstorage.
>
> rel-storage =
>      type mysql
>      db userZodb
>      user zodbuser
>      passwd innzop
>      blob-dir ${buildout:var-dir}/blobstorage
>      shared-blob-dir false
>
> # shared blobs are much faster if we're on the same server.
> # if not, turn it off.
> shared-blob = off
>
> Then I ran the buildout. All was built successfully. After starting the
> zeoserver, I got this error from the client:
>
> user@server:~/Plone433-dev/zeocluster3$ ./bin/zeoserver start

There is no need to start the zeoserver anymore.  zeoserver is only for
filestorage.

> ..
> daemon process started, pid=35136
> user@server:~/Plone433-dev/zeocluster3$ ./bin/client1 fg
> 2014-12-17 14:50:31 INFO ZServer HTTP server started at Wed Dec 17 14:50:31 2014
>      Hostname: 0.0.0.0
>      Port: 9180
> 2014-12-17 14:50:32 INFO Products.PloneFormGen gpg_subprocess initialized, using
> /usr/bin/gpg
> Traceback (most recent call last):
>    File "/home/user/Plone433-dev/zeocluster3/parts/client1/bin/interpreter", line
> 289, in <module>
>      exec(compile(__file__f.read(), __file__, "exec"))
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/run.py",
> line 76, in <module>
>      run()
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/run.py",
> line 22, in run
>      starter.prepare()
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/__init__.py",
> line 86, in prepare
>      self.startZope()
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/__init__.py",
> line 262, in startZope
>      Zope2.startup()
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/__init__.py",
> line 47, in startup
>      _startup()
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/App/startup.py",
> line 81, in startup
>      DB = dbtab.getDatabase('/', is_root=1)
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py",
> line 287, in getDatabase
>      db = factory.open(name, self.databases)
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py",
> line 185, in open
>      DB = self.createDB(database_name, databases)
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py",
> line 182, in createDB
>      return ZODBDatabase.open(self, databases)
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/config.py",
> line 101, in open
>      storage = section.storage.open()
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/config.py",
> line 33, in open
>      return RelStorage(adapter, name=config.name, options=options)
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/storage.py",
> line 212, in __init__
>      self.blobhelper = BlobHelper(options=options, adapter=adapter)
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/blobhelper.py",
> line 118, in __init__
>      fshelper.create()
>    File
> "/home/user/Plone433-dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/blob.py",
> line 359, in create
>      (self.layout_name, self.base_dir, layout))
> ValueError: Directory layout `zeocache` selected for blob directory
> /home/user/Plone433-dev/zeocluster3/var/blobstorage/, but marker found for
> layout `bushy`

In var/blobstorage/ there is a tiny file .layout (note the dot at the
beginning) which marks the layout that is used.  Rename var/blobstorage
to something else as backup (for safety) and try again.

Or better: change the relstorage configuration in your buildout to point
to a directory with a name that indicates it is a cache:

   blob-dir ${buildout:var-dir}/blobcache

> Unfortunality I have no idea where the problem could be.
>
> I think the best would be to simply disable blobstorage completely. This way
> Plone would be forced to use filestorage respectively the Relstorage for the
> blobs. So I decided to comment out all lines defining "blob-storage =
> ${:var}/blobstorage". I thought Plone would now using Relstorage for the blobs
> too. But after the buildout there is still the following line in
> parts/zeoserver/etc/zeo.conf:
>
> blob-dir /home/openmind/Plone433-dev/zeocluster3/var/blobstorage

Don't use the zeoserver. :-)  MySQL has the data now and the zope
instance talks directly to it, instead of via a zeoserver.  Personally,
in the [instance] part of a buildout where I use RelStorage I have set
'zeo-client = false' just to be extra clear.  (Not really needed, as the
zeo-client and rel-storage options are mutually exclusive anyway.)

> Why want I do this? Because our customer wants only one big MySQL database which
> stores filestorage and blobstorage. So he can backup his system easier. I know
> this is not a good idea, but he wants so.

I have a similar client. :-)

Best,


--
Maurits van Rees: http://maurits.vanrees.org/
Zest Software: http://zestsoftware.nl

_______________________________________________
Setup mailing list
[hidden email]
https://lists.plone.org/mailman/listinfo/plone-setup