ragged blog

How to do pythonic list comprehensions with conditions in ruby? | One Big Library.

Somone was discussing this, I thought I’d post some examples:

I see the selects being written there attempting to operate on the variable. If this does do anything, it will be in-place on the array (or range) that you’re operating on. For an array, this is not really a problem, other than you probably didn’t want to do that:
[source:ruby]
>> a = (’a’..’z').to_a
=> ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"]
>> a.select do |l| l =~ /[aeiou]/ ? (l << ‘-mod’; l) : nil end
=> ["a-mod", "e-mod", "i-mod", "o-mod", "u-mod"]
>> a
=> ["a-mod", "b", "c", "d", "e-mod", "f", "g", "h", "i-mod", "j", "k", "l", "m", "n", "o-mod", "p", "q", "r", "s", "t", "u-mod", "v", "w", "x", "y", "z"]
[/source]
Obviously, you could make a copy of the array at this point, and it would solve the problem, at the obvious cost. See at the end for some examples of another way to achieve a similar effect in ‘one line’.

With a range, the outcome is different:
[source:ruby]
>> a = (’a’..’z')
=> “a”..”z”
>> a.select do |l| l =~ /[aeiou]/ ? (l << ‘-mod’; l) : nil end
=> ["a-mod"]
>> a
=> “a-mod”..”z”
[/source]
Why? because you’ve in-place modified the range. Now, what is an enumerator to do with a range, when you’ve modified it in-line? What is the next item from ‘a-mod’..’z’ from ‘a’?
‘a’ is no longer in the range.

If you want numeric iteration over a list, you first of all need a list!
[source:ruby]
>> r=[]; (’a’..’z').each {|v| r << v if v =~ /[aeiou]/}; r
=> ["a", "e", "i", "o", "u"]

>> y = []; for x in 0..10 do y << x * 3 if x % 2 == 0 end; y
=> [0, 6, 12, 18, 24, 30]
[/source]
The last ‘;var’ is only for irb to print the array we just created, you can remove it, or, if you really really want to, you could do this (might be suitable for a function)…
[source:ruby]
var = (y = []; for x in 0..10 do y << x * 3 if x % 2 == 0 end; y)
[/source]
There are some things in the ruby grammar I truly love, no matter how ‘useless’ they may seem.

None of this really offers you the pythonic syntax, but that’s not necessarily a bad thing.

The python syntax provides (in ruby terms) essentially two block passes. One before the for, and one after.

We can get closer to replicating this, but it still runs over the array twice (so the above solutions may be better), and the ordering of operation is different:
[source:ruby]
(0..10).select { |v| v % 2 == 0}.each {|v| v * 3}
[/source]

In comes inject…

[source:ruby]
>> (0..10).to_a.inject([]) do |a,x| a << x * 3 if x % 2 == 0; a end
=> [0, 6, 12, 18, 24, 30]
[/source]

djberg96: The Paradox of (Programming Language) Choice

Just keep the options down, and ensure as much compatibility as possible. Think of the Java VM and SDK options, people don’t complain too much about those differences because they don’t affect most code.

So someone has come up with a solution for linking Erlang to ruby. This could be potentially useful.

The original article: Calling Erlang from Ruby (Teaser)

The library is called Rebar (Ruby to Erlang Bridge And Runner). Rebar has two parts: an Erlang server and a Ruby client. The server listens for JSON-RPC requests, calls the specified Erlang function, and responds with a JSON-RPC response. The Ruby client object simply accepts any method name, captures it with method_missing, wraps everything up in a JSON-RPC request, fires it off, waits for the response, and then extracts the result out of the JSON-RPC response, returning it like a normal Ruby call.

Unfortunately this means that you’re going to be waiting for both RPC, parsing, routing, and method_missing in ruby. Thus the ‘teaser’ I would say.

Erlang also has some powers of reflection though. A better solution for some applications would be to publish a dsl between them for the particular application. Maybe even a full object dsl. Depending on how much meta-information can be made available to each side, this direction may get faster.

I think the most reliable way would involve generating the protocol API for the RPC from ruby written generators. This should have the advantage of adding a fully ruby DSL. I’m starting to discuss a design that looks more like ruby2erlang now though. I guess maybe it’s not /that/ hard.

Anyway, this could solve a number of problems relatively trivially for scaling networked apps, with erlang handling buffering and non-blocking high concurrency jam, and ruby based ‘consumers’ (which it’s worth noting could live anywhere, which is not completely uncommon for such high load scenarios).

Anyway, this is really just another thing for my already very long list of ideas I’d like to give some attention :)

[source:ruby]
def factorial n
fac = 1
1.upto n do |i|
fac *= i
end
fac
end

def factorial n
fac = 1
for i in 1..n do
fac = fac * i
end
fac
end

def factorial n
fac = 1
1.upto n { |i| fac *= i }
fac
end

(defun factorial (n)
(loop for i from 1 to n
for fac = 1 then (* fac i)
finally (return fac)))
[/source]

This was a little bit of random inspiration I got from this little javascript file which _why wrote:
Ruby for Windows in Under 1K.

I thought that was pretty cool… So I made it a bit cooler, and now propose some ideas I’ll be working on if I can find the time…

When I stumbled upon this stranded little baby bear in the woods I thought it looked kinda cute. Unfortunately this little bear had no real motivation to execute it’s cuddly little plans. Seeing him so close to growing into a strong beast which might aid my hunting, I felt compelled to provide him with a spice of one-step motivation (before I redirected his focus in order to utilise him for my own devious plans of forest domination).

So anyway, the result was a nasty little .bat file in 640 bytes, which simply echos try.js and calls it. This version also hard codes support for EOT to exit.

Opps, I winked.

Anyway… time for some real web page nastiness with a wide line of code:

[source:ruby]@echo tryurl=”http://tryruby.hobix.com/irb?cmd=”;w=WScript;o=w.StdOut;i=w.StdIn;function wget(url,hdrs){var http=w.CreateObject(”Microsoft.XMLHTTP”);http.open(”GET”,url,false);for(var k in hdrs)http.setRequestHeader(k,hdrs[k]);http.send(”");return http.responseText;}sess=wget(tryurl+escape(”!INIT!IRB!”),{});w.Echo(”Interactive Ruby ready. (send ^D to exit)”);o.Write(”>> “);i.Read(0);while(true){cmd=i.ReadLine();resp=”";ps=”>> “;if(cmd==”\004″)break;if(cmd){resp=wget(tryurl+escape(cmd).replace(/\+/g,’%%2B’),{’Cookie’: ‘_session_id=’+sess});ps=”..”;if(resp){o.Write(resp);ps=”>>”;}}o.Write(ps+” “);}>t.js
@cscript//Nologo t.js&del t.js[/source]
- That’s 640 Bytes, with some extra documentation and an extra char on the prompt! I’m sure it could be shorter, but I don’t care. That’s damn near half a kB, and the block size of most all devices is larger than that these days (devices like, don’t even think this small, dude).

And the tryruby.bat

Now, on to some not so original and dangerous thinking…

The tool as it stood in a .js file was a bit cumbersome. You’d have to load up a cmd, get into the right dir, and run cscript on it. Now, you can just carry this .bat round on a USB stick and get a ruby XML-RPC to …. the try ruby server …. ;-)

As another kinda funky side idea, you could carry your own one-time pad algorithm in another .bat based on this, processed via tryruby, along with a username and password to get into your own ruby processes.

It might be a crazy stupid idea, to some, but it’s also pretty damn useful. I mean, it’s a remote ruby shell, with NO dependencies except the OSes native scripting languages…. I don’t even necessarily need putty after this (no, I’m not entirely serious there, but not far off either).

As I say, I’ll be looking into this more if I get the time, as the ideas are very loose right now, however, I should imagine it won’t be too hard to attain some reasonable level of security. Together with a little DSL this could make a very neat (and immediately extensible over the wire) remote control or remote data recovery system, with no dependencies.

I’m sure it’s easy to make a similar thing for *nix, in fact, I have vague recollections of a .sh version of a similar tool lying around somewhere too.

So the plan:

1. Look at tryruby, sandbox and whatever else may be relevant.
2. Build in SSL support if it doesn’t work already.
3. Build an authentication mechanism (maybe some crazy extras here)
4. Load up a server somewhere safe (maybe, in it’s own xen vm :) )
5. Give some of those goony linux sysadmins wet dreams

If I build it up to anything really useful, I’ll release it. I’m obviously concerned about security, but equally, in this case (or this direction of use) we’re intending to open a hole. I just want some strong authentication and traffic protection before I’d let anyone else ‘go wild’.

Does anyone use anything like this in ruby (considering how many of these kinds of things are out there)?

Maybe I’ll refactor this post to be more succinct or make more sense later. But for now, I don’t care. It’s a hack and I think it’s cool.

Keep up the good work _why.

Update: Found a bug in the code, pointed out from the original fix mentioned in comment 3. The string replace was actually not running across the whole string, even in _why original version. The the first argument to replace actually needs to be /\+/g rather than ‘+’.

Thanks to anyone who submits any more bug reports, as this really isn’t something I’m spending time on right now :D

Here’s the discussions:

1. Should Database Manage The Meaning?
2. Choose a single layer of cleverness

Here’s my opinion / comments:

• “I think the cognitive discrepancy lies in equating RDBMS with an operating system. Nothing justifies that parallel.” (1) - He’s comparing access restrictions, and it’s true to say that in rails, there are no restrictions on your applications management of your data.
• “If we step back and look at what RDBMS is, we’ll no doubt be able to conclude that, as its name suggests (i.e. Relational Database Management System), it is a system that specializes in managing the data in a relational fashion. Nothing more.” (1) - But what you use *contains* an RDBMS, it’s not an RDBMS in it’s own right. It’s an RDBMS, plus DDL, plus various many other features.
• “Your procedural ambitions will bear no fruit and you’ll have to pry that logic from my dead, cold object-oriented hands.” (2) - My DDL hands ARE object oriented. You poor poor poor mysql user.
• If you utilise the features of your DB, then your domain model can be reduced to ONE function, and you can treat your DB as a remote procedure call system.
• Using a Database system as an arbritrary store of FLAT data is SLOW AS A SNAIL. So why oh why oh why would you use a database for this? Like seriously, why not use flat files for storing each of your activerecord models? If they don’t do anything but store data, then files are truly more appropriate, and can be implemented A LOT faster than any DB link will achieve.
• “Folks, it’s important to keep in mind that it manages the data, not the MEANING of the data!” (1) - Excuse me? my constraints and stored procedures completely manage the meaning of the RPC calls I make to the DB. The strict object oriented type system I use there means that my db objects are in fact carrying more solid meaning than ANY single ruby class or object, whose meaning may change depending on who ducks…
• “A word processor (such as the much maligned MS Word, or a much nicer WordPress, for example) specializes in managing words. It does not specialize in managing the meaning of the words.” (1) - Except most people rely on their grammar checkers, automatic contents page creation, and so on. These things are features missing from activerecord, and available in my RDBMS
• “it uses the RDBMS to manage that data in a relational fashion.” (1) - Oh really, because selecting data from one table at a time is relation use and relational management? No. That’s row based management, and is better suited to flat files.
• “You’d toss that piece of junk out the window in no time.” (1) - Unless actually you’re a lawyer, and by not structuring your document properly you will loose a case, or make a to-be infamous mistake in a lawful document. In fact, your abstraction is as poor as that to which you have been responding. Data consistency is about rigid unbendable rules. Artistic creation is a different process and belongs in a different tier.
• “Why should we tolerate RDBMS opinions on our data?” (1) - Why should be tolerate rails lack of regard for data consistency and relationship complexity? You claim that in this case opinionation is bad, and yet…. “Active Record is opinionated software, just like the rest of Rails” (2).

Get a grip you lot. So you built something that is really simple stupid, and that makes it easy to coerce and teach to others. That DOES NOT mean you should go to such great lengths to protect immature and unproven ideas as if they are some kind of evangelical movement to enlightenment. You have your place, but it’s not appropriate for everyone.

When two people talking about the same thing completely contradict each other, and make the same abstraction-al mistakes as their opponents, I really start to doubt the sincerity of their claims. Realistically there are a lot of scenarios where you need to ensure that your data is made available through an RPC api, and that your data is held consistently. I know this is out of the scope of the pure web development domain, but it’s still in the application domain. To ignore this is genuinely short sighted, not loud thinking. If you weren’t ignoring it, there would be better support for other methodologies of building or discovering data models in rails, rather than making those of us that like the functionality, build large wrappers to compensate for poor extensibility.

N.B. Unfinished and very directly opinionated right now. More to come and more succinct and backed up examples to come.

I recently wrote this in an e-mail discussion about the project I’ve recently been working on. It’s really a rant, but it’s also true.

In building our GPS tracking system, I had to build a parser for the TETRA device. Fundamentally, this is akin to building a modem driver for a serial modem really. Now, I decided long ago that we would go the route of supporting *nix and win32 accross the board, and we would make choices to facilitate this. This gives us a few things - the ability to develop on windows and unix simultaneously - the ability to provide no-cost and low maintenance solutions for long-term services.
My discovery simply verified my fears of the reality of kernel maturity accross the board.
- Even in the POSIX definition of RS232 I/O there is no well defined standard for accessing RS232. Well, the sad fact actually is, it’s in the POSIX standard, but other things are not. This means it’s inconcistent with most implementations, in some cases in trivial ways, others quite destrutive ways.
- The only clean *nix method was via kernel sysread and syswrite calls, through a POSIX termios interface for charecter flow control. This is a place where true POSIX complience lacks in many OSes IMRE.
- The Win32 API, whilst it provides a more standardised syscall set, requires direct kernel communication, as all of the available libraries are subject to severe limitations.

The result is:
- Unlike most other devices, there is no abstraction away from opening a kernel handle (win32) or filedescriptor (posix/unix) which is reliable and available (no standard ‘driver’ as such, maybe the modern definition of driver).
- Every single API that is available is buggy in some way, this includes API’s for: Perl, Python, Ruby, C/C++ with common OS libs, C#, VB, PHP, BASIC, Assembler. Now that’s a pretty extensive list.
- The above was true of all OSes too.

Some conclusions:
- Most programmers are very very poor indeed. (Many of the win32 libs were going to cause a segfault about every 950 days or so (by statistical calculation), at the data rate they were being used (low))
- Most software never sees proper testing in real world type states, prior to production. - Time and bit-level chaos are hard to test against in poor designs, and designs often miss these possibilities.
- Most people are unable to properly read or utilise standards.

UI oriented design sometimes aids this, and in the case of daemons, I/O oriented design can lead to some high claims too (in C mind.): http://cr.yp.to/qmail/guarantee.html

The modern approach is to provide simple agnostic interfaces between things and to seperate user facing code from system facing code. Whilst this paradigm shift is growing, it hasn’t reached the kernel or OS core libs yet, and may not for another 20 years. This industry moves fast though, and I’m hoping to help one day too, so it may come sooner.

So Ruby is being a wonderful language to write in (seriously more enjoyable than most) and is giving me access (cleanly) to advanced language features I’ve wanted from other places. (Although I don’t hack in Smalltalk, Scheme or Lisp, I understand alot of features which I have desired in a more practically accessible language for a long time). Ruby seems to have slowed me down in a few places, and lower level string handling is one such place. As with the last post, I’ve been working with ASCII hex tuples and the support is a bit weaker than one might hope (this is not uncommon anyway, and as always there’s a balence of length of API vs functionality).

I’ve had to write what is effectively the reverse of the previous posting, but in the ‘guise’ of a simple xor hashing function (suitable for NMEA parsing btw ;):

[source:ruby]
def xorchecksum(str)
xor = str[0] str[1..-1].each_byte { |c| xor ^= c }
return res.chr.unpack(”H8″)[0].to_s
end
[/source]
This maybe could be more efficient somehow else, but this works. Certainly as I’m producing code that is to be production level and yet never going to be overloaded with data, this is currently suitable. I’m aware that there may be one particular efficiency increase if I try to somehow not allocate object space so often, but at this time I think it’s relatively close to that optimal already. The excess syntax required feels a bit limiting, abusing Duck Typing a little to get the correct results back.

This and reading 1959 page ETSI documentation is my current day-in day-out, but we’re getting close to release when I’ll probably be posting here a little note on what we’ve been hermitting for the last couple of months.

My current project required a simple operation to perform that took me some time to find in ruby. I’ll put a copy here so hopefully it’s of use to someone:
[source:ruby]str.scan(/../).each { | tuple | puts tuple.hex.chr }[/source]
So this will convert Hexidecimal tuples into ASCII literals, like follows:
[source:ruby]str=”48656C6C6F”
str.scan(/../).each { | tuple | puts tuple.hex.chr }
H
e
l
l
o
=> ["48", "65", "6C", "6C", "6F"][/source]
As I say, I couldn’t find this anywhere else, so enjoy.

So I’ve been learning Ruby very rapidly for a new project, and well, it’s all about watching programmers say ‘Whoops’.

Normally, it’s tappy tappy, GRR, ARG, !”£$”£%”!^”U*%^*!£%$”£$^”$*, crash bang, whoosh, SMASH and some dead hardware.

Ruby is more like ‘Whoops’, fixed, ‘Whoops’, fixed, ‘Whoops’, ‘Finished’.

Glorious.