1995 Apple World Wide Developers Conference Kaleida Labs ScriptX DreamScape Demo
Apple Worldwide Developers Conference
Don Hopkins, Kaleida Labs
DreamScape is a constructive experience built on ScriptX, based on rooms and objects, that combines a whole lot of different metaphors, and demonstrates many of the neat things about ScriptX. DreamScape is also known an Actualized Fantasy Screen Waster (not to be confused with a Virtual Reality Screen Saver).
DreamScape Demo Video
DreamScape Demo Transcript
This is a transcript of a demo of DreamScape, written in ScriptX, at the Apple Worldwide Developers Conference, on May 11 1995. The demo, given by Don Hopkins of Kaleida Labs, was run on a PowerBook 540c, and the video from the powerbook was resampled and recorded on video tape. My demo didn’t crash (but it came really close), so I didn’t have to do push-ups (which was the official rule). The images herein are frames grabbed from the video tape of the demo.
Warnings and Disclaimers
PUBLIC NOTICE AS REQUIRED BY LAW: Any use of this product, in any manner whatsoever, will increase the amount of disorder in the universe. Although no liability is implied herein, the consumer is warned that this process will ultimately lead to the heat death of the universe.
To make any sense of this, you should realize that it’s live improvisational performance programming art. The graphical and audio artwork are just ugly placeholder “programmer art”. The references to “great content” are laughably ironic!
I didn’t have an art department at my disposal, and I have terrible graphics design skills, but I didn’t let that stop me, because that wasn’t the point!
The art is in how it works and what it does and how I use it, not how it looks or sounds.
It was meant to inspire ScriptX developers as well as myself, and it led to me to later develop iLoci and MediaGraph.
Thanks for Mark Weiser’s Advice
I’ll quote a discussion with Mark Weiser about my demo video tape that made his wife run screaming from the room:
Mark 8/3/91>Yes, saw the tape. A work of art! Great audio track. I sat mesmerized.
Don 8/6/91> Oh great, it worked!
Mark 8/3/91> (My wife ran screaming from the room, on the other hand).
Don 8/6/91> If I could *only* understand why people react that way, I think it
would help alot…Mark 8/6/91>I think it helped me to know you, and to understand what a good time you were having making the tape, and that *there was something* there to get into. So I let me self get involved, and I had a good time too.
Therefore, I think you should bill the tapes as artworks. That gives people the right frame of mind — ok, its not computer technology, its personal, its a thing to get into. It also helps them have an open mind — people are used to looking at weird art and having their mind’s bent. Could be great!
-mark
Thanks for Craig Hubley’s Advice
DreamScape was inspired by Craig Hubley’s sound advice and vision that it’s possible to empower every user to play around and be an artist with their computer.
DreamScape Demo Transcript
I hope my screen saver hasn't zapped me. Is that on video? I think I have to turn on video mirroring here.
OK, I'm going to demonstrate DreamScape which is (oh, there we go) DreamScape is a constructive experience built on ScriptX, that combines a whole lot of different metaphors, and demonstrates many of the neat things about ScriptX.
It's based on rooms and objects. We're in a room here, and here's an object, a part. The rooms are connected together in a map.
This is my "user representation" [avatar], this is Bill, Bill Jellavich.
We click on him and we get a map, centered on where we are.
So we're in this room here, and I can actually edit this map, disconnect it, and I could, say, give this presentation backwards by starting over here.
I can move around and and I can bounce off the side, and then go off the edge into a different room.
I find things in the room. Like here's a flower.
And we have some animated objects. And this is a weird spiral thing.
And we can do constructive things with them, like stick them together, and, you know, build Barrel of Monkeys or Mr. Potato Head type things.
And some of these things have behaviors, like this is for duplicating things, so I'll just drag it over the flower, boom, and it fissions, just makes another flower.
Fortunately, the head is magic, because the head represents you, and if you duplicated you, and then went out of the room, what would you do? So I had to put some special code there.
Another weird thing: you can do this astral traveling, by clicking on the head, and going to another room, and then you don't have the head, and you can't get out, so don't lose your head, remember where you left it.
Great content, how do you get great content?
You got to import it from different sources, like Director files, aiff things, and bitmaps. Then you can, like…
This is intended to provide some templates for people to import common things without having to program.
Like these records here, you can just drop an audio file into place, and just drop a new image of the record, and you’ve got a new one.
We’re debating about whether anybody will remember how to use this user interface here.
We’ve got timelines and things, and animations.
A lot of these animations are defined in Director.
The same way that Director doesn’t make Photoshop obsolete, you do your images in Photoshop, and then load them into Director, and animate them.
We load our animations into Director, and put registration points on them, and I’ve made a special Director importer.
Before you go insane, I’ll change the record.
Stop that. And I’ll stick this one on.
I’ll do a Michael Jordan here.
I have ESP: I can use wind to blow objects around.
Woah, down there. Woah.
Ok. So each of these rooms here is an object that we can distribute on the web, as a separate “title container”.
We can use ScriptX as a web “helper application”, and point to a ScriptX title container, and you click on it, it downloads, and it starts running, and loads into whatever you’re doing.
So you could get more records and things, and people could publish interesting interactive objects.
Woops. OK. This is really easy to use.
Oh wait. Hmm. So, umm. OK.
The thing is, this is not a Killer App. It’s a nurturing environment.
We want to give creative people an environment in which to plant their seeds, a fertile ground, instead of a Killer App.
You can give an artist Director, and say, do some neat graphics, put some registration points down, and load it into ScriptX.
Here’s where the Astral Traveling comes in handy. Hmmm!
These objects can be authoring tools.
Little mini authoring tools.
Like this map you could consider a way to change, to move things around.
This is the smooching user interface, you just kiss them up together.
Dynamic extensibility. We have distributed multimedia publishing.
You can take these and make neat presentations on the web. Oh, my screen fell in, oh no!
This butterfly here, you could call that an agent if you want.
Woah, ok, he just flew away, let’s follow him, see where he went.
Here we are down by the beach in San Francisco.
We can grab the butterfly and move him around, and get over by the head there, come on.
It’s hard giving demos with unpredictable life forms.
I don’t claim it’s very intelligent.
Come on, you worm with wings. Thank you. Ok.
So now the butterfly’s in control of the presentation.
So, where are we going to go.
Woah, he’s brought us to this volcano. So let’s get rid of him, whoosh.
So these things here are robot parts, and I can grab them, and stick them together, like that, just kind of like Legos, sort of.
And these again were defined in Director. I’ll pop over to Director.
This is how you define them. It’s just: you have the animations in the timeline, and you have the names you want to give them up here as labels, and you can put sounds in, and then these things here are the registraton points: neck, left shoulder, left leg.
So you just take the cast, you put in rectangles for the places you want to plug things in, you just stick them down on the stage, and just do that for every frame, and then you’ve got yourself an animation, and then you load it into the Director importer.
And then you can do things like, just stick stuff together.
Let’s see… Or you could make a more conventional robot. This is your avitar construction kit.
Once you get them all together, he’s very happy.
So, what is this good for?
As Negroponte says, Direct Manipulation is only good for driving and sex.
We need some other interfaces, and this is a multiple metaphor system.
What in the world just happened?
Umm, I think the gravity warped.
That little N, the NetScape icon you saw there, is a dynamically loaded part that gives us an interface to NetScape.
I’m running NetScape locally, and now I have the ability to open a URL, call it “scriptx:hello”.
OK, this is talking to the ScriptX locally, that’s dynamically generating HTML, and I can just click on the button, and that links back to “scriptx:help”.
All these services are registered.
There’s a class browser.
It just looks at the classes that are in memory, and generates a web page to describe it, that you wouldn’t want to have to write yourself.
And all this nasty HTML. Come on BBEdit!
And all the HTML just links back into ScriptX, and it’s all programatically generated.
And the cool thing is that these HTML things are like Collections, they’re very easy to manipulate in ScriptX.
You just say “for i in interestingThings do collect a link to i as WebBulletedList”. And you’ve got this bulleted list. Here’s a LinearCollection.
This is a tool here that uses NetScape as an interface. When this DreamScape thing runs, it dynamically loads a C extension that lets it talk to NetScape.
Aaah, stop that. Woops, ow.
And then the DreamScape thing registers an inspector for this application. We’ll just hit “dream”. There’s a bunch of other neat little services.
Here are the rooms defined in DreamScape, and I think we’re in Mobile, and we’ll just click on that.
It loads the gif exporter, writes out a little thumbnail of the room that we’re in, tells us the neighbors, to the left is a volcano, and then the contents, and this is a nested list here, a nested bulleted list, reflecting the structure.
It’s an AnimatedPuppetPart, which is the body.
The cool thing is that we can click on these links, this room browser has links to object browsers, and that’s the AnimatedPuppetPart, he’s in room Mobile, where I came from, he has children left leg, neck, right leg.
And the cool thing, is that here are some forms to edit his elasticity. We’ll make that guy two, and then hit “Submit”.
So that just changes that guy’s elasticity.
Let’s see. Woah. Goes off the side, and his body hits. Boom.
So I’ll just start ping-ponging back and forth here, unless I do something.
But, woah, stop stop! Woah! No! OK, wait wait wait! My demo’s falling apart!
OK, wait, stop! Stop! Stop! Arrgh! Stop! Do I have to do push-ups? Ok, wait wait wait wait. Ok.
Ok, so we’ll make a little thing here.
You can spend hours doing this.
I think there’s a game in here somewhere.
Come on. Right there.
Ok, well anyway, I’ve changed the structure, and I’ll go back to NetScape, and then link back to the room here, and you’ll see the change reflected in the outline, and there’s the new nesting there.
And I could change the gravity of the room, or the friction.
Or the cool thing is I could click on this image here.
This is like an image map, but it’s talking to a real program, and it can look at the coordinates of the mouse click, and then go to the real thing that you clicked on, and it just exported an interlaced gif, and here’s the whole room.
If these things don’t move around, I’ll just click on Bill’s head here. Woah, come on NetScape. Where’s my cursor? Woah.
I’ll just click on the robot thing here, and that just goes back to the room. Of course if it had moved in the meantime, I wouldn’t get it.
That may be the problem. Let me give you a peek under the hood of ScriptX. It’s exporting this giant gif, I think.
The cool thing is that this is using an interface to NetScape that also works with the SpyGlass browser, and also works on Windows.
So this is the log of the ScriptX server, and the cool thing is that we can find out stuff, we have a very intimate interface with the browser, better than just being a helper app.
It tells us every time it goes to some URL, so we can follow that and build a map.
We can register for a protocol, that’s why we have this “scriptx:” thing here.
This is the key to writing really cool authoring tools for NetScape.
This is like, uh, I mean, you know, it’s a map, boom, like that, why don’t we just say “Save as ScriptX”, and export a whole network of web pages with image maps and things like that.
That’s the next step I’m talking this towards.
And this NetScape will sort of be like the previewing engine for this multimedia web that you’re going to generate.
And the cool thing is that this multimedia web that you’re going to generate with this authoring tool I’m working on, will be able to have links in it to the actual ScriptX title containers, that represent the objects from which it was generated, that you can download.
You can go window shopping, look at the storefronts, the gif files that were generated, that show here’s what’s on sale today, and then links to real objects that you can take out for test drives.
That’s why I want to do this authoring tool in ScriptX.
Ok, whoops, woah, ok! Landed in a tree.
I want artists to be able to use these tools to create things.
One neat idea I have that I haven’t quite finished yet is being able to edit trees.
Think, this is isomorphic to an outliner, this little tree here is like an outliner, and I can just move parts of the outline around by direct manipulation.
And if I went into NetScape, I’d see, where am I, woops.
Not only can I, oh wait.
Go to the room he was in, cause that’s an old link.
And I can put these puppet parts together to build a tree, or a program could generate them by looking at the links I’m following in NetScape, or just doing whatever.
I’ll go inspect this room.
This brings the whole map up.
And that’s an image map that I can click on, and if I moved the map around, I would of course see the updated map.
Here we go. The Mac’s not really great at process switching.
See, this is the outline. Imagine just taking tree structures and representing them as these puppet graphics.
And there are cool things that you can put on the tips of the things. Like this here is a seed.
You can think of this as sort of a vertical painting tool, for painting with dandylions, and you pick the flower, and bring it around, and it leaves little petals around.
This is just an object that was also brought in through the director importer.
The director importer can handle many really neat special formats, so if you’re going to be doing a whole lot of something that requires media in a certain way, you can come up with a specialized director importer than can handle that.
So you get the idea.
You can get a set of objects from one place, and a set of objects from another place, and combine them in really unexpected ways.
So, ok, here’s, ha ha ha, I don’t know what this guy’s name is, he’s dude.
This is some fabulous stuff. We have not only, oh!
The timeline metaphor, I forgot to show the timeline, I’ll do that next.
But we have things that are driven by rules and reactions and events.
Like you can click on this cheese and it wafts.
He says “Zed is dead baby, Zed is dead”.
These animations can go on at the same time, this is not click and wait, there’s a simulation going on here.
This is Pavlov’s lab here.
This is a prototype, but it’s going to be dynamically configured by the object that you drag into it.
We can say, like, when the dude sleeps, screams, explodes, sleeps, talks, wafts, no, when the cheese wafts, the dude sleeps, explodes, screams, screams.
Yeah, all right.
And then you go (click) waft the cheese. (The cheese wafts but the dude doesn’t scream.) Hey.
When the cheese wafts … (Victor in the audience yells “Ring the Bell!”)
Oh, the bell! Thank you! Ding!
When the cheese wafts, the guy screams!
He was not a plant, I forgot, I really did forget. So. (Click.)
(Cheese wafts.)
(Guy screams ayyyeyeyeyay!)
See why I didn’t want audio?
If that was on the loudspeaker, it would be too much.
So, you can have authoring tools, you can dynamically load them.
Let’s go to that place where the robot parts were.
So, the idea is, well what is a tool, what is an authoring tool?
I’d like to blur the lines, you know.
Umm, I mean, woah, I’m lost, excuse me.
You have wind. You can press down on the background and drag, so you can appear to have ESP.
OK, woah, stop, stop, ok. Now, ok, let’s stop these guys, and get rid of that guy.
The cool thing is that it’s all multithreaded, you know, you just set something on its way, and go do something else, it’s gestural.
Ok, now import a tool at runtime. ScriptX dream parts.
So we have zillions of neat little parts.
The cheese, the flower, timeline.
If this was an adventure game, you wouldn’t want to include the map, cause that would let people cheat, or might let people cheat, maybe they’d have to build their own map or something.
So this is a timeline that could be sychronized with some music that’s playing in the room, or something like that.
I can drag one of these objects onto the timeline, and it makes a little icon for it, and then when the time goes over it, it triggers it.
Imagine a game where you’ve got a lot of weird Rube Goldberg-esque contraptions that you can stick together, and you have to put them together in some way, and then arrange a timeline, so that it goes out and grabs, and shoots a basket or something, or you know, who knows?
We’re just trying to make an open ended framework where people can come up with stuff like that, and do these demos.
They’re about to show me the bomb, aren’t they? Don’t worry, it’s not going to crash.
The future direction of this is as a web authoring tool, and we’ll be putting this on the net as soon as I can get it sort of finished.
And I got a whole lot of work done on it night before last right before the first talks we did, so we’re making progress.
OK, I’ll let the next person go now. Thanks.
DreamScape Documentation
iLoci Demo
DreamScape is the predecessor of iLoci.
MediaGraph Demo
iLoci is the predecessor of MediaGraph.
