Pennysmalls

cannot find code object on disk

Fri, 06 Jan 2012 02:22:49 -0800

A quick note for anyone googling this particular problem:

Getting "cannot find code object on disk" when compiling with latest XCode with code signing turned on?

Hard to find the solution on the net, but eventually I figured it out - latest xcode does not have all the tools bundled yet. Do this:

ln -s /usr/bin/codesign_allocate /Applications/Xcode.app/Contents/Developer/usr/bin/codesign_allocate

Permalink | Leave a comment »

Change Space grows up

Sat, 19 Nov 2011 02:13:00 -0800

At the end of August I posted about Change Space.app, which seeks to replace the sorely missed 2D grid spaces feature that was in Snow Leopard for those unlucky enough to be using OSX Lion.

Now it's two and a half months and over 50 commits later, and I have just released version 2.2.0. I think the app is getting pretty usable.

The last post I mentioned how I was using MacRuby to write the app, and indeed this was a pretty quick and easy way to get it off the ground.

But there are a couple of issues with MacRuby. Firstly, you need to set your Xcode up to include the MacRuby framework and runtime in the .app file otherwise users will have to install it themselves. And the runtime adds about 10Mb to your app size.

Secondly, when integrating with other libraries and certain calls to the Cocoa libraries and other APIs you find yourself having to write bridge code where the structures you are dealing with do not map easily to ruby objects.

Given these issues, I decided to rewrite the app completely in Objective C.

By mid October I had the App running nicely in the new language, and it was coming in at about half a meg as opposed to just under ten.

Some great libraries helped with the features I needed. Shortcutrecorder provided an ability to change the global hotkeys.

Change Space also displays the current desktop number in the status bar at the top of your screen.

At this time, rather than fade the desktop number out as I had been doing previously, the number was shown in the status bar permanently by polling the system for which desktop was active each second. I was to find a better way to do this though - it turns out that there is a notification for desktop change, so I can just subscribe to that and remove the polling altogether.

NSNotificationCenter *notCenter;
    notCenter = [[NSWorkspace sharedWorkspace] notificationCenter];
    [notCenter addObserver:self
               selector:@selector(workspaceObserver:)
               name:NSWorkspaceActiveSpaceDidChangeNotification object:nil];

I also wanted to add central screen notifications of the direction of movement between your spaces, and which you went from and to.

I started off with a simple arrow, but it would be far nicer to see an entire grid of the right dimensions. Step in Harry Groover, who provided code for a very nice looking grid. I added the arrow, and we had a really great looking result.

Next up I wanted the desktops to move in the right direction.

The importan APIs are defined like this:

extern OSStatus CGSNewTransition(const CGSConnection cid, 
  const CGSTransitionSpec* spec, int *pTransitionHandle);        
extern OSStatus CGSInvokeTransition(const CGSConnection cid, 
  int transitionHandle, float duration);

The reverse engineering has already been done for us by previous apps such as DesktopManager, so the enum documentation can be found.

Not everything seems to work in Lion, but we have enough to work with. Calling CGSNewTransition will freeze the existing desktop, and wait for a transition to be invoked.

CGSInvokeTransition will perform the specified transition.

Unfortunately I do not have (as yet) a way to disable the standard left or right lion transitions, so when I want the desktop to move up or down I need to wait for the left or right transition to happen between creating the new transition and invoking it. Experimentation tells me that 0.3s is a long enough wait for that.

So the vertical transitions have a slight delay before they activate, but I think it's worth it to preserve illusion of a grid.

For those who can't stand the extra wait, I added a preference option to allow these transitions to be turned off.

So now, I think Change Space.app has come of age.

There are still a few problems to iron out, in particular the space 1 issue (I must use the AppleScript bridge to navigate to it, which is slower and is affected by which modifier keys you are holding down).

I hope to have this problem licked soon though.

Download Change Space.app v2.2.0 here.

Permalink | Leave a comment »

Fixing the lack of 2D spaces in Lion

Wed, 31 Aug 2011 13:43:00 -0700

Tl;dr See my Change Space app

In OSX Lion Apple dispensed with the simple and effective 2D spaces grids many people loved and relied on.

In its place, there is the more dynamic concept of 'Desktops', which can be created by the system on the fly, for instance when running a full screen app. Also, the system can re-order the desktops according to your use.

These dynamic features are of little use to power users, who know where their apps and windows are, and like to navigate fast between them. A common layout is a 3x3 grid - then from the central space you can reach 4 others with just one keypress. Very useful for flicking between your code editor, and the browser, and perhaps mail, or iTunes for instance.

Even the animation when moving between desktops on Lion is slower than with the old Spaces system, making it really feel like a big step backwards for usability.

I'm not the only who feels this way - see stackoverflow here http://stackoverflow.com/questions/6768684/osx-lion-applescript-how-to-get-cu... ... and there's even a facebook page here https://www.facebook.com/pages/Save-Spaces/216451795053936

While many still hope that Apple will bring back the old Spaces system as an option, in the mean time something needed to be done.

Having found some clues on the stackoverflow post mentioned above, I thought I'd have a go at writing an app to do the directional switching.

Applescript

If there's one thing I hate it's applescript. But, it has its uses. The first thing to do was to try a few little scripts to see if I could get to a particular desktop.

To run this from the command line I needed this kind of incantation

arch -i386 osascript changespace.applescript 4

And it worked - if you don't happen to have any modifier keys other than control pressed, and if you don't try to go to any desktop above number 4

Not good enough, but the information will come in use later.

CFNotificationCenter

There was a gist linked from the stackoverflow post that contains a few lines of code that can both find the id number of a desktop (they are no longer numbered sequentially - see here for an interesting post about this), and can move you to a particular desktop by sending a message to CFNotificationCenter.

This is almost everything we need. There is a wrinkle though - it appears that desktop 1 is special, and this code cannot navigate to it.

Also desktop 1 always returns the id number 1 (as opposed to some random 5 or 6 digit number).

Putting it together

At first, I made a short shell script for each direction, that ran the compiled C code or some applescript, and hooked those scripts up to Alfred.app using the Global Hotkeys feature (which is very handy by the way).

This was a disaster. Alfred starts a shell to run the shell script in, but this inteferes with the current showing desktop (because I have my terminals set to appear on desktop 5). So although I could see movement, it wasn't useful.

So I need an app

What better time to have a play with macruby?

There are plenty of resources on the net to help you get started.

So I got macruby installed, and got myself set up in Xcode with a new app, Change Space.app.

One of the really cool things about macruby is the total integration with Objective C message passing. Everything just works together with Ruby method calls like magic. Really, it's great - skim through this introduction if you are interested.

As I don't know a way of directly calling C functions from macruby, I wrote a bridge class that calls them for me using Objective C messages that I can send from macruby.

So with this bridge over to the existing C code for changing and identifying spaces, I'm part of the way there.

For switching to desktop 1 I still need to send a keypress - after some investigation I found that the ScriptingBridge framework is the answer.

Here's how I send a keypress using it:

It was rather opaque how to send the control down, and to be honest I have no idea how you would send something like control-shift. But no matter, we have what we need.

I still needed to hook the hotkeys. And I have davedelong to thank for the answer, DDHotKeyCenter.

For the next part, I wanted a status bar icon like the old spaces app. This gist was helpful, and the code is simple to adapt.

As people have different layouts for their spaces, I needed to set some preferences. In fact you can set a preferences panel up in the interface builder without much effort, see this guide from Apple.

To access the settings from macruby is simple:


  defaults = NSUserDefaultsController.sharedUserDefaultsController
layout = defaults.values.valueForKey("gridLayout")

I added a few icons, and Bob's your uncle - Change Space.app v1.

Celebrate with Champagne

Well, maybe not (oh ok just a sip). The app is not perfect in any way - the biggest niggle is that holding the shift key down after commanding a move to space 1 causes that movement to fail. Hopefully that can be solved.

But there are other niggles - it would be really nice if we could make the desktops move in the right directions. And if we could get notification of when the active desktop changes then the desktop number indicator on the status bar could be there at all times.

Anyway, the beauty of github is anyone can contribute - so please go ahead and fix it!

Permalink | Leave a comment »

Rails 3 with jumploader

Wed, 06 Apr 2011 06:37:22 -0700

Java applets went out of fashion some years ago, and you don't see them very often now. But they still can be useful. We had a file upload problem that was not solved by plupload, uploadify, swfupload or any other common solutions. The answer was JumpLoader - with this we can upload files bigger than 2Gb, and it's got a host of other nice options and features. (Yeh, I know it may not be a good idea to try to send files >2Gb through a browser, but that's what we need to offer - and at least we can arrange to resume failures.) The downside is that it needs to be code-signed if you do not want your users to see a self signed certificate warning when they run it for the first time. Those certificates are quite expensive ($200 per year depending). But assuming that's not a problem, let's dive in and get it working with Rails 3. There are two things you need to send when posting a file upload to rails - the request forgery protection token in with the post params, and the cookie that identifies you in the header. Fortunately, JumpLoader offers us both possibilities. I've installed the java files in public/java. So this is how my applet tag looks, in HAML: And the resulting tag in the HTML output: Here we have set the requestProperties to the session cookie, which ensures it is sent with the request headers. We also tell the applet to 'fireAppletInitialized' which is a javascript function. Here is the javascript from the head of the document, in ERB this time: This sets the token in a parameter that will be sent with the posted file. Now the upload should pass the csrf check. Note the 1 second delay before setting the parameter in the applet. This is because I was seeing hangs in Safari when setting the parameter directly from the appletInitialized function.

Permalink | Leave a comment »

Much faster Rails tests if you use MyISAM

Thu, 05 Nov 2009 10:14:33 -0800

Here's a little insight I had the other day. If you use MyISAM tables, and there are various performance related reasons you might, then you've been stuck having to turn transactional fixtures off in your tests. But unless you are using some special feature of MyISAM that is not present in InnoDB, then why not use InnoDB tables in your test database? I wrote a simple plugin that has a rake task that clones the development database to the test while changing the ENGINE to InnoDB. You just run the task, turn transational fixtures on in test/test_helper.rb, and you're off. Result: our tests used to take 8.5 minutes to run. Now they take just 3 minutes. That's about 65% less time than it used to take. The plugin is here: http://github.com/sdsykes/fast_fixture

Permalink | Leave a comment »

FastImageInline - inline images in your html code

Wed, 23 Sep 2009 21:19:16 -0700

You may not know this, and I didn't until recently, but you can place images directly in your img tags. Not just the address of them, but the whole binary data - base 64 encoded. The technique is based on the Data URI scheme. Take a look at the source of Google News. Scroll down a bit and you should start to see image tags that contain base64 encoded data (well you will if you are on IE8 or another make of browser - not IE7). That's what I'm talking about. The technique is particularly suitable if you have have small images that change often and you do not need the browser to cache them. In this case, the saved http connection and fetch that inlining the images affords you is a performance win. In case you find it useful I have extended my FastImage series with FastImage Inline, which is a gem and rails plugin that will take care of inlining your images. It's simple to use - just like image_tag there is now a helper method inline_image_tag (and inline_image_path for image_path). Example

inline_image_tag("bullet.gif")

Result for request from a data-uri capable browser:

Result for a non-capable browser (eg IE7 or below):

Installation Note that the FastImage gem must be installed first, check the requirements section below. As a Rails plugin

./script/plugin install git://github.com/sdsykes/fastimage_inline.git

As a Gem

sudo gem install sdsykes-fastimage_inline -s http://gems.github.com

Install the gem as above, and configure it in your environment.rb file as below:

...
  Rails::Initializer.run do |config|
    ...
    config.gem "sdsykes-fastimage_inline", :lib=>"fastimage_inline"
    ...
  end
  ...

Requirements * FastImage http://github.com/sdsykes/fastimage Browser support All modern browsers support this technique except for IE versions 7 and below. This is still a major segment of the market of course, but as IE users migrate to IE 8 this will become less of a problem. FastImage Inline uses a simple browser detection mechanism by looking at the user agent string. If the browser is known to not have support, or if we do not recognise it at all, we serve a normal image tag which includes the path to the image file in the src attribute. But if we know the browser can handle it, we send the image inline, and the browser won't need to fetch it separately. Limits Reportedly IE8 will not handle data strings longer than 32k bytes. But it is probably unwise to inline images this big anyway. Google news serves images that are up to about 3.5k in length, and this seems a reasonable approach. However, FastImage Inline does not enforce any particular constraints, it is for you to decide. FastImage Inline does not cache the images it has read - so every time an image is sent it will be read from disk. This feature may be added in a later release. Conclusion Inlining images is not for everyone, but it's a useful technique in your toolbox for optimising delivery of certain kinds of pages or content. For more information check the comprehensive list of advantages and disadvantages on the Data URI scheme wikipedia page.

Permalink | Leave a comment »

Recent code - FastImage resize, Scrooge and Read From Slave

Tue, 14 Jul 2009 14:57:09 -0700

A roundup of some of my projects that may be of interest: 1. FastImage Resize This builds on my work on FastImage to provide an image resize facility. The resize code calls libgd to do the work of resampling and resizing the image - this is a library that is very likely to be already installed on your system if it is some flavour of unix / linux or even OSX. And if not, it is very easy to install. This is a light and simple option if you don't wish to install heavier libraries such as RMagick (which relies on ImageMagick or GraphicsMagick) or ImageScience (which relies on FreeImage). 2. Scrooge This is a plugin and gem to optimise queries to the database based on a learning algorithm that looks at how the results of each query are used. I worked on this with Lourens Naudé earlier this year, and I will shortly make a minor release with a few further optimisations and tests. Try this if your database is slowing you down, but also see slim-attributes if you are using MySQL. 3. Read From Slave A gem to force your database reads to a slave database while your writes go to the master. It's fast and simple, it works a treat, and we have it in production use.

Permalink | Leave a comment »

FastImage finds image dimensions fast using minimal resources

Thu, 11 Jun 2009 11:37:36 -0700

I just released a gem to find image dimensions and type information fast. I have previously done some work in this area, but this is a much more comprehensive solution, and fixes problems with certain jpegs. FastImage finds the size or type of an image given its uri by fetching as little as needed The problem Your app needs to find the size or type of an image. This could be for adding width and height attributes to an image tag, for adjusting layouts or overlays to fit an image or any other of dozens of reasons. But the image is not locally stored – it’s on another asset server, or in the cloud – at Amazon S3 for example. You don’t want to download the entire image to your app server – it could be many tens of kilobytes, or even megabytes just to get this information. For most image types, the size of the image is simply stored at the start of the file. For JPEG files it’s a little bit more complex, but even so you do not need to fetch most of the image to find the size. FastImage does this minimal fetch for image types GIF, JPEG, PNG and BMP. And it doesn’t rely on installing external libraries such as RMagick (which relies on ImageMagick or GraphicsMagick) or ImageScience (which relies on FreeImage). You only need supply the uri, and FastImage will do the rest. Examples

require'fastimage'

FastImage.size("http://stephensykes.com/images/ss.com_x.gif")

=> [266, 56]  # width, height

FastImage.type("http://stephensykes.com/images/pngimage")

=> :png

Installation Gem

sudo gem install sdsykes-fastimage -s http://gems.github.com

Rails Install the gem as above, and configure it in your environment.rb file as below:

...

Rails::Initializer.run do |config|

...

config.gem "sdsykes-fastimage", :lib=>"fastimage"

...

end

...

Then you’re off – just use FastImage.size() and FastImage.type() in your code as in the examples. Documentation http://rdoc.info/projects/sdsykes/fastimage

Permalink | Leave a comment »

Ferret on Ruby 1.9.1

Tue, 24 Mar 2009 09:47:00 -0700

I took the trouble to port ferret to ruby 1.9.1 yesterday evening. I have it working on my mac. Here's a gem for you to try - I have labelled it 0.11.6.19. If you use it let me know how it runs, but it's at your own risk, I haven't extensively tested it. [UPDATE: this gem has been updated 5th April 2009 - please test. There is also a fork at github] I've made mostly simple changes in the code:

Changed all struct RString -> ptr to use the RSTRING_PTR macro, except for cases where it was being used to add items to an array where rb_ary_store was used.
Changed all struct RString -> len to use the RSTRING_LEN macro
Changed all struct RArray -> ptr to use the RARRAY_PTR macro
Changed all struct RArray -> len to use the RARRAY_LEN macro
Removed manual adjustment of the len member of RArray. In fact ruby 1.9 stores small arrays of 3 items or less differently from larger ones, and this adds complexity. It is better to use the rb_ary_store method which will use the correct pointer and will keep the length in sync with the number of items in the array.
Changed all struct RHash -> tbl to ntbl
Removed references to rb_thread_critical
Removed 4th argument from calls to rb_cvar_set
Included ruby/re.h and not regex.h, and altered tokenizer code to correctly use the new regexp library
Included ruby/st.h and not st.h
Some other minor changes to error messages formats causing compiler warnings

By the way, acts_as_ferret also runs with some very minor surgery, Thomas von Deyen has a fork here.

Permalink | Leave a comment »

Rails 2.3 breakage and fixage

Wed, 04 Mar 2009 14:49:53 -0800

Rails 2.3 will be with us soon, so I took the time to update our app to be compatible. It's a reasonably large app (26,000 LOC), so there's bound to be some issues. The first thing to notice is that the PStore store for sessions has completely gone away. This means that saying something like config.action_controller.session_store = :p_store in your environment file will no longer work. We don't use the cookie store because our sessions can get bigger than the 4k limit in certain circumstances. So we use the memcache store on the production machines, and pstore on the dev and test machines. And we can't do that any more, which is a shame as it worked well - particularly with Hongli Lai's improvements. The remaining options are DRb, Memcached, or SQL. We didn't want to add complexity to our environments, so none of those looked attractive. So we ended up rewriting some of our code so that the cookie store would be usable in most cases. We'll keep memcache as the store on the production systems though. Talking of memcache, it seems we now need to include require 'memcache' in our production.rb file. It's not automatically loaded before we want to configure it. The rest of the problems weren't with the app itself, but with the incredible amount of failed and erroring tests due to changes in the Rails testing system. Firstly all the unit tests were not even running because they all inherited from Test::Unit::TestCase. Nowerdays they need to inherit from ActiveSupport::TestCase, and this is necessary in Rails 2.3. Also make sure your test_helper.rb opens the right class:

ENV["RAILS_ENV"] = "test"
requireFile.expand_path(File.dirname(__FILE__)+"/../config/environment")
require'test_help'

class ActiveSupport::TestCase

...

Next, if you were using assert_valid model_item you must change this to assert model_item.valid?, see here. No deprecation warning in 2.2 that it would be removed, but never mind, the fix is quite easy. We have tests for our routing. They live in the unit tests - it's handy to test all the routes in one place. But in 2.3 the assert_routing method has disappeared. In fact it's just not automatically available in unit tests any more, you can retrieve it by doing this in your test class:

include ActionController::Assertions::RoutingAssertions

But the routing assertion also needs clean_backtrace which seems to be part of the ActionController::TestCase. We opted to just define it in test_helper.rb (for a quick fix, just add this code):

  def clean_backtrace(&block)

    yield

  rescue ActiveSupport::TestCase::Assertion => error

    framework_path = Regexp.new(File.expand_path(

                                     "#{File.dirname(__FILE__)}/assertions"))

    error.backtrace.reject! {|line| File.expand_path(line) =~ framework_path }

    raise

  end

We also test cookies in functional tests, and the usage has changed in 2.3. So you'll need to check through those. If you send multipart emails and have file fixtures (of the expected email contents) to test them, we noticed that instead of just saying Content-Type: text/plain in the header before the mime encoded parts, we now get Content-Type: text/plain; charset=iso-8859-1. Those need to be edited. Finally, if you are using assert_select_email in your tests for your mailer classes, you will find it is also no longer available. The fast solution is to put include ActionController::Assertions::SelectorAssertions in your mailer test class. We have worked around some of the issues presented to us with minimul changes to our code. It seems like Rails is encouraging us to organise our tests differently, particularly where functionality in ActionController::Assertions is no longer automatically available to unit tests. Working around this feels somewhat unclean, so we'll take a look again whether tests should be moved or rewritten once the dust has settled on 2.3.

Permalink | Leave a comment »

Using acts as ferret with phusion passenger / mod_rails

Mon, 02 Mar 2009 20:59:31 -0800

The passenger manual makes it clear that you need to close and reestablish your connections to things like memcached after it forks to avoid inadventently sharing file handles. The reason is well and clearly explained there. The api to do this is simple - just place this kind of code in your environment.rb file:

if defined?(PhusionPassenger)

  PhusionPassenger.on_event(:starting_worker_process) do |forked|

    if forked

      # We're in smart spawning mode.

      -- reestablish connections --

    else

      # We're in conservative spawning mode. We don't need to do anything.

    end

  end

end

All well and good, but how exactly do you reestablish those connections? In our case we have to deal with memcached and ferret (with ferret running in a DRb server via the acts_as_ferret plugin). Memcached is dead easy:

CACHE.reset

Ferret not so easy. It turns out that DRb has no in-built way to close its pool of connections. So a monkey patch is the only thing to do. I was inspired by some code you can find here. But since we want to blindly close all the connections, our case is simpler:

classDRb::DRbConn
defself.close_all
@mutex.synchronizedo
@pool.each{|c| c.close}
@pool = []
end
end
end

  DRb::DRbConn.close_all

DRb will happily reconnect by itself when needed after its connection pool has been emptied. Putting it all together, it looks like this:

ifdefined?(PhusionPassenger)
# monkey patch drb so we can close its connections
classDRb::DRbConn
defself.close_all
@mutex.synchronizedo
@pool.each{|c| c.close}
@pool = []
end
end
end

  PhusionPassenger.on_event(:starting_worker_process) do |forked|

    if forked

      # We're in smart spawning mode.

      CACHE.reset  # memcached

      DRb::DRbConn.close_all  # ferret

    else

      # We're in conservative spawning mode. We don't need to do anything.

    end

  end

end

Permalink | Leave a comment »

Yum refuses to update

Tue, 24 Feb 2009 11:04:56 -0800

I happens that I wish to upgrade apache, and I know there is an update available. So, I follow the advice given:

# yum update httpd
Setting up Update Process
Could not find update match for httpd
No Packages marked for Update

Ok, that's not working. Let's try this:

# yum clean all
Cleaning up Everything
# yum update httpd
fedora                    100% |=========================| 2.1 kB    00:00     
primary.sqlite.bz2        100% |=========================| 5.8 MB    00:02     
updates                   100% |=========================| 2.6 kB    00:00     
primary.sqlite.bz2        100% |=========================| 2.2 kB    00:00     
Setting up Update Process
Could not find update match for httpd
No Packages marked for Update

No dice.
I browsed the repo, and sure enough the update is there. Why won't yum find it?
Eventually after a bunch of googling, I found this answer.

# yum upgrade
Setting up Upgrade Process
Resolving Dependencies
--> Running transaction check
---> Package fedora-release.noarch 0:8-6.transition set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================
 Package                 Arch       Version          Repository        Size 
=============================================================================
Updating:
 fedora-release          noarch     8-6.transition   updates            31 k

Transaction Summary
=============================================================================
Install      0 Package(s)         
Update       1 Package(s)         
Remove       0 Package(s)         

Total download size: 31 k
Is this ok [y/N]: y
Downloading Packages:
(1/1): fedora-release-8-6 100% |=========================|  31 kB    00:00     
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Updating  : fedora-release               ######################### [1/2] 
  Cleanup   : fedora-release               ######################### [2/2] 

Updated: fedora-release.noarch 0:8-6.transition
Complete!
[root@c100586 yum.repos.d]# yum update httpd
updates-newkey            100% |=========================| 2.3 kB    00:00     
primary.sqlite.bz2        100% |=========================| 4.6 MB    00:01     
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 0:2.2.9-1.fc8 set to be updated
--> Processing Dependency: httpd-tools = 2.2.9-1.fc8 for package: httpd
--> Processing Dependency: httpd = 2.2.6-3 for package: httpd-manual
--> Processing Dependency: httpd = 2.2.6-3 for package: mod_ssl
--> Running transaction check
---> Package httpd-tools.x86_64 0:2.2.9-1.fc8 set to be updated
---> Package mod_ssl.x86_64 1:2.2.9-1.fc8 set to be updated
---> Package httpd-manual.x86_64 0:2.2.9-1.fc8 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================
 Package                 Arch       Version          Repository        Size 
=============================================================================
Updating:
 httpd                   x86_64     2.2.9-1.fc8      updates-newkey    983 k
Updating for dependencies:
 httpd-manual            x86_64     2.2.9-1.fc8      updates-newkey    832 k
 httpd-tools             x86_64     2.2.9-1.fc8      updates-newkey     68 k
 mod_ssl                 x86_64     1:2.2.9-1.fc8    updates-newkey     86 k

Transaction Summary
=============================================================================
Install      0 Package(s)         
Update       4 Package(s)         
Remove       0 Package(s)         

Total download size: 1.9 M
Is this ok [y/N]:

That's better.
Really, it was totally unclear from any documentation that I could find that I would need to run yum upgrade.

Permalink | Leave a comment »

Breakage and fixage in Rails 2.2

Sat, 29 Nov 2008 16:54:28 -0800

Finally our app is completely Rails 2.2 ready. Some quick notes on some issues and things that needed to be fixed: 1. Default error messages This call is no good any more

ActiveRecord::Errors.default_error_messages

Use this instead:

I18n.translate('activerecord.errors.messages')

2. Use ActiveSupport::Inflector rather than Inflector The warning tells you all you need to know:

DEPRECATION WARNING: Inflector is deprecated! 
Use ActiveSupport::Inflector instead.

3. Integration tests are broken if you are not using the cookie store See here for details. If you are seeing "NoMethodError: You have a nil object when you didn't expect it!" inexplicably from your integration tests, then this could be the issue. I ended up placing this in environments/test.rb, even though it should not be needed:

config.action_controller.session = { :session_key => "_myapp_session", 
  :secret => "some secret phrase of at least 30 characters" }

4. Use of string keys in assert_redirected_to in tests no longer works Consider this code:

assert_redirected_to "host"=>"foobar.com", 
  "action"=>"something", "controller"=>"hw"

It used to work, but in rails 2.2 it does not. You need to use symbols for the keys, like this:

assert_redirected_to :host=>"foobar.com", 
  :action=>"something", :controller=>"hw"

I think the change was made in this commit. 5. Render_partial is gone If you are still using render_partial in places, you should replace it with render :partial=>"partial_name" 6. HAML is not yet compatible with Rails 2.2 There is a problem in HAML that causes output from calls to content_tag (and other tag helpers) to be lost. See this thread for details. If you use HAML I do not recommend upgrading to rails 2.2 until this issue has been sorted out. However, it'll probably be fixed in a day or two, and the thread I linked to contains details of the patch I used if you need to fix it before that. 7. Components are deprecated We had to rewrite some old code that was using components. It actually wasn't too much effort in the end, and the resulting refactoring was an improvement anyway. -- As ever our tests were extremely valuable during this process. Once HAML is sorted out we will be upgrading our production server to Rails 2.2, so hopefully that will be in a day or so.

Permalink | Leave a comment »

Backup now

Fri, 21 Nov 2008 16:59:41 -0800

Ever seen a shattered hard disk?

It happened to a friend. So backup your data. Now!

Permalink | Leave a comment »

Slim-Attributes v0.5.0 released

Tue, 14 Oct 2008 18:57:17 -0700

I just released a new version of slim-attributes. There are some small speed gains and some other minor changes from 0.4.1, but there are no big changes. Read more about slim-attributes at the slim-attributes homepage, or read on below. Introduction Slim-attributes is a small patch to the ActiveRecord Mysql adaptor that stops rails from immediately making ruby strings from the column names and data from your database queries. Because you probably don't need them all! So ruby strings are lazily created on demand - it's faster and uses less memory. And it drops directly in, requiring only the installation of a gem and adding 1 line to environment.rb. Measuring with just ActiveRecord code - fetching stuff from the database - we see anything up to a 50% (or more) speed increase, but it really depends on your system and environment, and what you are doing with the results from the database. The more columns your tables have, the better the improvement will likely be. Measure your own system and send me the results! Installation Try:

gem install slim-attributes -- --with-mysql-config

or:

gem install slim-attributes

then add this to environment.rb:

require 'slim_attributes'

Description Normally the mysql adaptor in Rails returns a hash of the data returned from the database, one hash per active record object returned by the query. The routine that generates these hashes is called all_hashes, and this is what we replace. The reason for overriding all_hashes is threefold:

making a hash of each and every row returned from the database is slow
rails makes frozen copies of each column name string (for the keys) which results in a great many strings which are not really needed
we observe that it's not often that all the fields of rows fetched from the database are actually used

So this is an alternative implementation of all_hashes that returns a 'fake hash' which contains a hash of the column names (the same hash of names is used for every row), and also contains the row data in an area memcpy'd directly from the mysql API (which is much faster than creating ruby strings). The field contents are then instantiated into Ruby strings on demand - ruby strings are only made if you need them - when you ask for a particular attribute from the model object. Note that if you always look at all the columns when you fetch data from the database then this won't necessarily be faster that the unpatched mysql adapter. But it won't be much slower either, and we do expect that most times not all the columns from a result set are accessed. Future development I speculate that further speed gains might be had through keeping the mysql result objects from mysql-ruby around, and not copying the data from them at all until it is needed. However, mysql-ruby limits the non freed result sets to just 20 before calling GC.start, so surgery inside mysql-ruby would be required to achieve this.

Permalink | Leave a comment »

Splitting models into several smaller files in Rails

Mon, 06 Oct 2008 20:52:38 -0700

I was reading what Paul Barry had to say about splitting models into smaller files. It resonated with me a little - some of our models are approaching 1000 lines. But I felt the name 'concerned_with' did not fully / appropriately describe what is being done, and that there should be an easier way than having to specify every file to be required. So I ended up modifying the code to be a little easier to use. If you place it in an initializer (i.e. in a file in your initializers directory), then you can specify in your model that you wish to require all the files from a subdirectory of the same name as the model. So if you model is called Customer, then the model file is customer.rb. Now you can also have a subdirectory called customer that contains further files containing model code. In the original model class file you should add require_class_subdirectory to it, like this:

class Customer
  require_class_subdirectory
...
end

This will cause all the files in the subdirectory to be required. In each file in the subdirectory you should open the model class like so:

class Customer
  def something  # you can cut/paste code in from the main model file
  end
...
end

The filenames you use don't matter - in the above case it could be 'something.rb' for instance. So, to recap, your main class file customer.rb has 'require_class_subdirectory' added to it. You create a folder called 'customer' in your models directory, and place some .rb files in there. In each of those files you re-open the class ('class Customer') and place code there just as if you were writing into the main class file. This allows you to separate code according to function within a model, and to keep file sizes manageable. Here is the code to put in the initializer:

class << ActiveRecord::Base
  def require_class_subdirectory
    ActiveSupport::Dependencies.load_paths.select{|lp| lp =~ /app\/models/}.each do |path|
      Dir["#{path}/#{name.underscore}/*.rb"].each do |filename|
        require_dependency "#{name.underscore}/#{File.basename(filename)}"
      end
    end
  end
end

Other approaches to this problem are possible. In particular it may be feasible to patch or hook the constant missing mechanism in rails to automatically load the files in the subdirectory, which would remove the need for the require_class_subdirectory line in your main model file. Finally, not even everyone thinks this is a problem that needs to be solved. My colleague who uses Aptana says it has a good outline mode that means it's easier to work with one large file for a model than lots of smaller ones. In Textmate I find the smaller files easier.

Permalink | Leave a comment »

How to disable the query cache in rails 2.1

Wed, 01 Oct 2008 06:54:39 -0700

If you need to disable the query cache in rails, it's not particularly easy to do that. There is some discussion about it (dated March 2008) here. Although you can turn all caching off, and you can turn the query cache off explicitly in your code using uncached, there isn't a way to turn just the query cache off globally at configuration time. So, then, here's the monkey patch you need (tested on Rails 2.1.1). Although this is not particularly optimal (in that some query caching related code is still called), it will work. You can put this at the bottom of your environment.rb somewhere, or even better put it in its own file in the initializers directory (e.g. query_cached_off.rb).

module ActiveRecord
  module ConnectionAdapters
    module QueryCache
      private
      def cache_sql(sql)
        yield
      end
    end
  end
end

Permalink | Leave a comment »

Find jpeg dimensions fast in pure Ruby, no image library needed

Tue, 19 Aug 2008 07:05:43 -0700

It so happens that in our application we need to find the dimensions of many images that do not reside on our rails application server. I have previously written about how to use the GD library to find image sizes, but this requires fetching the whole file and having it available in the local file system. Fortunately we know all these files are jpegs, and if you check the jpeg documentation you can see that the dimensions of an image are normally contained near the start, which is ideal. We will just fetch enough of the image to get the dimensions, and no more. How can we do that? The get method from Net::HTTP will do the trick - given a block it will yield each packet of data from the remote server as it arrives. And fetching over http makes this method very general - we can size any jpeg from anywhere using this code. Parsing the returned data for the dimensions requires a simple state machine which will break from the http get block once the dimensions are located. So I ended up with a small class to encapsulate the dimension information from the jpeg, JpegDimensions. Here's how to use it:

jpg_info = JpegDimensions.new("http://somewhere.co.abc/an_image.jpg")
jpg_info.height  # is the height
jpg_info.width  # is the width

And the best thing is that only a small part of the image will be fetched, saving time and bandwith. Here's the code

require 'net/http'

class JpegDimensions
  attr_reader :width, :height

  def initialize(image_path)
    @uri_split = URI.split(image_path)
    find_jpeg_size
  end

  def find_jpeg_size
    begin
      http = Net::HTTP.new(@uri_split[2], @uri_split[3])
      state = 0
      http.get(@uri_split[5]) do |str|  # this yields strings as each packet arrives
        str.each_byte do |b|
          state = case state
          when 0
            b == 0xFF ? 1 : 0
          when 1
            b >= 0xC0 && b <= 0xC3 ? 2 : 0
          when 2
            3
          when 3
            4
          when 4
            5
          when 5
            @height = b * 256
            6
          when 6
            @height += b
            7
          when 7
            @width = b * 256
            8
          when 8
            @width += b
            break
          end
        end
        break if state == 8  # don't need to fetch any more of the image
      end
    rescue Exception=>e
      # I do nothing here, but you can do something more useful with the exception if required
    end
  end
end

Permalink | Leave a comment »

Rails on Ruby 1.8.7

Sun, 29 Jun 2008 16:24:37 -0700

Although Ruby 1.8.7 is not officially recommended yet for Ruby on Rails, it does in fact work fine with Rails 2.1. And version p22 contains all the latest security fixes. For what it's worth, I can confirm that not only are our apps working well under 1.8.7 (including ferret and over 30,000 lines of app code), but they are consuming / leaking considerably less memory. (They still do leak, but the rate is much reduced.) So upgrade now. Beware of one issue - we had to clean up our ERB code to not contain comments that are not specifically marked inside their own tags like <%# comment %> because of different handling in 1.8.7.

Permalink | Leave a comment »

Poetic code

Wed, 25 Jun 2008 16:09:22 -0700

Thanks to Óscar Toledo for pointing out to me that my 2004 IOCCC winner is mentioned in this interesting book (in French). It's mentioned on page 53 with some nice commentary.

Permalink | Leave a comment »