REBOL and the Shell Author: Jeff Kreis Date: Sep, 2000 (orig. published at unixreview.com) Updated: Jun, 2001 ===Introduction When I first met my wife, Maida, she had an old black and white television that was one of the greatest works of hacking I'd ever seen. This TV was on its last leg, but it served us well for our entertainment needs. In place of an antenna, there was a coat hanger. The volume was stuck at an obnoxiously high level, so Maida had taped a large wad of Kleenex over the speaker. To suit her esthetic sensibilities, she had also decorated the old TV with bright paint. We were so used to our peculiar television that we lived happily, oblivious to its obvious deficiencies. Here in the year 2000, the majority of the programming languages in popular use come to us from the 1970s and 1980s. These languages are between 20 and 30 years old, and, like Maida's old TV, they have been patched relentlessly with ingenious little hacks to stave off impending obsolescence. When I first encountered REBOL, I knew immediately that I was looking at the slickest TV set to hit the shelves in decades. I considered the various programming languages I'd been working with, and I suddenly saw the Kleenex, the coat hanger, and the big manual channel dial. I realized that I had had very low expectations for programming languages, and had accepted a lot of their deficiencies without much question. For example, I thought that the more malleable a programming language was, the more complex it had to be. That assumption was based on my experience with a number of scripting languages that provided a lot of capability, but no straightforward way to do most things. One of REBOL's philosophies is that "simple things should be simple to do". What is a "simple" task, though? The low expectations we have of programming languages leads us to think that "simple" means adding numbers or maybe some string manipulations. In REBOL, the term "simple" becomes broader. Ask yourself, how easy is it (in the language of your choice) to read a Web page and save it to a local file? In REBOL, that would simply be: write %index.html read http://www.example.com How easy is it to read a pop mailbox and write the contents to an ftp site? In REBOL, it's a single one-liner: write ftp://usr:pass@example.com read pop://usr:pass@example.com How easy is it to send every message from a pop mailbox to another email address? In REBOL, it is as simple as saying it: foreach message read pop://usr:pass@exaple.com [ send somebody@example.com message ] There are endless examples of the ease with which REBOL accomplishes tasks you would not ordinarily consider to be "simple," especially tasks related to networking. Part of this ease stems from REBOL's use of descriptive "smart" datatypes that represent ubiquitous forms of data found in this Internet age. In REBOL, you represent email addresses as you would expect: name@example.com URLs are represented exactly as you are already used to seeing them: http://www.example.com/file ftp://ftp.example.edu/file finger:// whois:// nntp:// pop:// When you READ, WRITE, or OPEN a URL, a handler for that URL's protocol takes care of doing the right thing. (In REBOL, you can even implement your own protocol handlers that are represented by a unique URL.) You can write money as you would expect: money? $1'000'000 == true Typing MONEY? into REBOL reports that one million dollars is in fact money. (The double equal sign is REBOL's default result indicator when working in a console session.) All REBOL datatypes have similar type-checking functions. You can write Internet addresses (tuples) as you would expect: 127.22.33.1 * 1.0.0.1 == 127.0.0.1 As you can see, you can perform mathematical operations on tuples. Tuples also represent RGB values in REBOL. You can directly represent HTML or XML tags: tag? == true print [ "Ref" ] and deal with time and dates: 1-Jan-2000 + 10 == 11-Jan-2000 now + 0:30:00 == 5-Sep-2000/10:19:14-8:00 In the low-expectation programming universe, datatypes are nothing more than thinly dressed representations of memory cells. By adhering to such low expectations, all our networking programs will begin the same way -- first, re-implement URLs, emails, tuples, time, tags, etc... Since using REBOL, I've raised my expectations for programming languages. The examples above are based on REBOL/core, the original Internet Messaging language. A newer version of REBOL that is being developed (and free to download) is REBOL/view, which combines all the capabilities of REBOL/core but adds the ability to put GUIs on the screen. GUIs represent a huge problem domain in which REBOL's "simple things should be simple to do" philosophy really shines. How hard should it be to put up a window with a green background, an image, some yellow text, a red button that sends your mom some email, and another red button that will quit the program? In most graphically capable languages or GUI toolkits, this task would be far from easy, and it could take at least a day to get something working. In REBOL/view it is simple: view layout [ backdrop green image %my-mom.jpg text "Hi mom!" yellow button red "Send" [send mom@example.com "Hi mom!"] button "Quit" [quit] red ] You might notice that I put the word "red" for the "Quit" button at the end instead of right after the word "button" as on the previous line. That's fine. REBOL's layout dialect can figure it out. These examples are intended to demonstrate the philosophy which has guided the design of REBOL and to provide a high-level introduction to the language. ---Getting Started with REBOL If you're interested in trying REBOL, first download it for your platform of choice from: http://www.rebol.com/developer.html If you have a recent RedHat CD (6.1 or higher), REBOL is on the applications CD. However, the most recent version will always be available on the REBOL site. REBOL currently runs on 42 different platforms, and the single binary runs in sizes between 200k and 400k, depending on the platform. REBOL/view is slightly larger than REBOL/core. Note that REBOL Technologies also releases experimental versions of REBOL containing the latest fixes and newly added features. When you have retrieved the REBOL package, unpack it. Start REBOL (./rebol) and type help. You will be given instructions on the HELP function, which you can use to learn what various REBOL functions do. As a self-documenting language, REBOL will help you learn REBOL. Instead of the word HELP, you can also use the question mark. Try the following in a REBOL shell: ? find This will print out information about REBOL's find function, including what arguments it takes and what refinements it uses. A "refinement" is like a switch that indicates a refinement in a function's behavior. Refinements are a unique aspect of REBOL. For example, find normally just marches through a series from the starting position forward until it finds what you're looking for: find "This is a string" "is" == "is a string" Above, find found the string "is" and returned the string from that position forward. A refined behavior would be to find something from the end backwards, and this is accomplished by the find function refinement /last: find/last "This is a string" "s" == "string" Above, find worked from the end of the string backwards until it found the string "s," returning the string from that position forward. In addition to refinements, there are only a few other concepts to become familiar with. Those concepts include the "block" (REBOL's universal dynamic container), series (data that is ordered), words, and values (words can refer to values). At this point, you've already learned a lot about the language. To continue your study, you can refer to the following resources. Further documentation is continually being added. There are two books currently available on REBOL, and they can be obtained from various online bookstores and from: http://www.rebolpress.com/ Other documentation for REBOL can be found online at: http://www.rebol.com/developer.html As you get into REBOL, you might want to join the REBOL mailing list. This list is very newbie friendly, and most questions are usually answered in a matter of hours by one of many REBOL gurus. To join, send an empty email to rebol-request@REBOL.com, with the word "subscribe" on the subject line, without the quotes. (To unsubscribe, follow the same procedure but using the word "unsubscribe", minus the quotes). Or to use a simple one liner to subscribe, enter this at the REBOL prompt. send rebol-request@rebol.com "subscribe" There are many tasks that you can easily accomplish using REBOL, but the remainder of this article will focus on the very specific task of creating useful shell scripts. A set of useful shell scripts can make life using UNIX much easier. REBOL may change your expectations of the power of shell scripts! ===REBOL and the Shell ---Making a REBOL Shell Script To turn a REBOL script into a general shell script, place the following "interpreter" line at the top of the script: #!/path/to/rebol -qs Replace /path/to/ with the path that leads to where REBOL is installed on your system. The interpreter line is the very first thing in the script, preceding the REBOL header, which follows on the next line. The flags -qs mean to run the script quiet (no banners are printed) and to run the script without security. REBOL's default security level allows it to write and read information from the network and to read from files, but REBOL will ask permission before writing files. When REBOL is running at the default security level with the -q flag, any attempt to write a file will cause the script to immediately quit. Therefore, lowering security on the interpreter line is necessary only if your script is going to write files. Keep in mind that ordinary shell scripts have no additional security measures built in at all. The examples included in this document specify the security lowering -s flag on the interpreter line only where necessary. To finish turning a REBOL script into a usable shell script, it must be executable and in your path. A convenient spot to put your scripts is in your personal bin/ directory off your home directory. You can check your PATH variable by doing echo $PATH to see that your bin/ directory is there. If it is not, you can add your bin/ directory to your path as follows. Under bash flavors: export PATH=$HOME/bin/:$PATH Under csh: setenv PATH=$HOME/bin/:$PATH The above line can go in your shell init script (.login, .profile, .cshrc, .bash_rc, etc.) to make the change permanent. Finally, to make the script executable: chmod u+x ~/bin/script Afterward, you should be able to type the name of your script at the shell and have REBOL execute it. In the examples below, we'll include the so called "interpreter" line pointing to /usr/local/bin/rebol and a small REBOL header. ---Strip control-Ms from All Files in a Directory I've seen hackers argue for days about the simplest way to strip the control-M's from text files that come from MS-DOS. There are certainly various terse ways to do this using sed, awk, and tr, but here is how to do #!/usr/local/bin/rebol -qs REBOL [Title: "Strip-Ms"] foreach file read change-dir system/options/path [ if not dir? file [write file read file] ] REBOL writes out files in your native operating system format, so by simply reading in a file and writing it out, those MS-DOS control-Ms are taken care of. Presuming you installed the above script in your path, you would visit a directory containing text files from DOS land and type in the name you gave your script (stripms, perhaps?). Be careful to only do this in a directory with text files. To read and write binary files, use READ/binary and WRITE/binary. This leads to an obvious improvement of the script: checking the suffixes of files before reading and writing them, only doing so for .txt or .r files, etc.. Notice that the script used system/options/path to determine where to go. There are a few different paths found in REBOL's system object: system/user/home -- User's home directory system/script/path -- Where the script is found system/options/path -- Where you were when you invoked the script ---Lowercase a Directory of Files Sometimes you wind up with a disk or an archive full of annoying uppercase file names. It's a snap to lowercase the whole directory with REBOL: #!/usr/local/bin/rebol -qs REBOL [Title: "Lowercase em"] foreach file read change-dir system/options/path [ rename file lowercase file ] The script above works a lot like the first script we looked at. The same approach works for many scripts that operate on the contents of a directory. ---Dump Web Page Text and Data Here's a small script to dump the plain text of a Web page: #!/usr/local/bin/rebol -q REBOL [Title: "Dump Web"] parse load/markup read to-url system/script/args [ some [tag! | set x string! (prin x)] ] If you called this script dweb, then you would invoke it from the command line like this: dweb http://www.rebol.com Changing this script to print out the links on a Web page is simple: #!/usr/local/bin/rebol -q REBOL [Title: "Dump Web links"] parse load/markup read to-url system/script/args [ some [ set x tag! ( if x: find/tail x "href=" [print trim/with form x {<">}] ) | string! ] ] Call it dlink, and try it out: dlink http://slashdot.org ---Passing Arguments to Your Script Arguments that you specify on the command line will be found in system/script/args. Here's a little script to help see what all these path, home, and args are all about: #!/usr/local/bin/rebol -q REBOL [Title: "Rebol paths and args"] foreach item [ system/user/home system/script/path system/options/path system/script/args ][ print [:item "==" mold item] ] Call the script rargs and put it in your bin/ directory. Change directories to /usr/local and type rargs foo bar. You should see something like this: system/user/home == %/home/yourname/ system/script/path == %/home/yourname/bin/ system/options/path == %/usr/local/ system/script/args == "foo bar" If no arguments were provided, the args would be set to NONE. As you see, if you supply more than one argument, they'll actually come in as a single string.[Ref 1] Command-line arguments undergo shell expansion first, so REBOL shell scripts can be composed with other shell expressions. For instance, using the rargs script from above, try these out: rargs * rargs $HOME rargs `ls /` ---Quick One-Liners Occasionally there is a need to do a quick math calculation or fast string manipulation. REBOL allows you to do these sorts of one-liners from the command line using the --do switch: rebol --do "print 2 * pi * 4" But it is convenient to have that process shortened a little. Call the next script reb: #!/usr/local/bin/rebol -qs REBOL [Title: "Reb"] random/seed now print do system/script/args First, the random number generator is seeded as a convenience for when we want to use our mini REBOL expression evaluator to produce random items. Then our script merely DOes the arguments. Using reb at the command line allows for many nice possibilities: reb "1 * 2" reb head reverse read %. reb pick x: read %. random length? x You may ask, What use is picking a random file? Perhaps each day 'xv' picks a random image from a directory and puts it on your root window, or perhaps you have a cron job that periodically changes your Web pages. reb random 100 reb send friend@somewhere.com '"Hey!!"' 0 Or more complex expressions: reb "do fib: func [n m][if n > 50 [quit] fib m probe n + m] 0 1" reb "0.0.0.255 + read dns://yahoo.com" reb `find . | grep -c foo`" * 4" The last example would print four times the number of files found recursively from the present directory that contain the string foo. Using shell quoting rules, you can mix shell substitutions with other REBOL expressions to evaluate. Much of the time, an argument can be provided as is, but some will need to be quoted to avoid unwanted shell substitutions. ---from via pop Sometimes you are on a system that does not have mail delivered locally and can only retrieve it via pop. I've always been a big fan of the from command (frm on some systems). It allows you to see a quick summary of your mail box, who each message is from, and the subject line. Here is an example of doing a from through pop, which also demonstrates how easily network protocols are manipulated in REBOL: #!/usr/local/bin/rebol -q REBOL [Title: "POP FROM"] me: "account" pass: ask/hide "Password? " server: "pop.server.dom" pop: open rejoin [pop:// me ":" pass "@" server] forall pop [ message: import-email first pop print [index? pop message/from tab message/subject] ] close pop This pop from script will first ask you for your password, hiding the keys you type. You could put your password right in the script, but it might not be safe in a multi-user environment. Of course, you need to plug in your account and your pop server in the appropriate places. If you have multiple pop accounts, this script can easily be made to from them all together. ---FTP public_html In the same vein as the previous script, here is another network convenience script. While working on a Web site, you have a local directory where you work on the source files. Periodically, you make changes to your local copies of the Web site files and you want to get them up to the remote Web server. There's no need to fire up a dedicated application to move those files; you can roll a quick little script like this: #!/usr/local/bin/rebol -qs REBOL [Title: "to-web"] me: "account" pass: ask/hide "Password? " server: "ftp.server.dom" foreach file read change-dir system/options/path [ if not dir? file [ write/binary rejoin [ ftp:// me ":" pass "@" server "/public_html/" file ] read/binary probe file ] ] The script is hard-wired to upload every file in the directory you call it from to public_html/ on your remote server. Creating a recursive directory upload is also fairly easy. If you end up getting network timeouts using ftp, try adding the following line to the beginning of your script: system/schemes/ftp/passive: true Setting passive to true in REBOL's ftp scheme will turn on REBOL's ftp passive mode. In passive mode, REBOL will connect back to the ftp site for the data transmission, instead of the other way around, as it normally goes. If you are behind a firewall, ftp servers will likely not be able to establish the data connection to you, hence the timeouts. ---A Simple Reminder Using REBOL/view, it is a snap to create a quick and dirty reminder program: #!/usr/local/bin/rebol -q REBOL [Title: "Reminder"] if found? args: system/script/args [ set [time message] load/next args wait time view layout [ text red (message) font [size: 20] ] ] Call the script remind and put it in bin/ directory. Here is an example of invoking the script: remind 2 update the database & Two seconds later, your message should pop up on your screen. If you want to wait a few hours, you'd specify the time in REBOL's time format: remind 4:00:00 it is four hours later & LOAD/next was used to extract the time value from from the first position in the arguments. ---Quickly View an Image REBOL/view can display jpg, gif, and bmp files. With REBOL/view you can make a simple script to view an image. When you click on the image or hit space bar while having the window in focus, the window will close. #!/usr/local/bin/rebol -q REBOL [Title: "View an image"] change-dir system/options/path if found? args: system/script/args [ view layout [image (to-file args) #" " [quit]] ] The #" " is the space character, and it defines a shortcut for the image. Following the space character is a block that is executed when that shortcut is pressed or when the image is clicked. REBOL/view's layout dialect is a very simple way to create dynamic GUIs. I called the above script eye. Moving into a directory with an image file, the script is invoked in the following way: eye picture.jpg As expected, the image pops up in a window. ---That's Just the Beginning The preceding examples demonstrate a few of REBOL's capabilities. It is very easy to build REBOL scripts for the shell that make your life much easier. The capabilities of this small-sized language may leave you wondering why you've settled for so much less from other larger languages. By the way, my wife and I now have an okay TV set, complete with a pair of bent up rabbit ears that have to be sprawled across the floor for decent reception. The moral is to always strive for improvement. Ref 1: Important Notes. Newer versions of REBOL provide each command line argument as a block of separate strings in system/options/args. system/script/args still provides all the arguments as a single string, making it handy for evaluation. At the time of this writing this change to the command line argument passing is only in the experimental versions, but it will be included in all future releases. There is a bug in the current released version of REBOL/core 2.3. The command-line arguments in system/script/args are presented as a block containing a single string of the arguments. To work around this bug, you may want to use one of the experimental releases until the next general release of REBOL, or use form or first on the system/script/args. form turns things into a string, and first gets the first thing out of the block. ] Copyright 2000 Jeff Kreis. All rights reserved. Jeff Kreis has a B.S. in computer science from the University of New Mexico. He is presently one of five developers contributing to the REBOL kernel and related projects, and has been doing so since shortly after the REBOL 2.0 release. Jeff also plays the Balalaika. He can be reached at jeff@rebol.com. ###