Phantom: a publishing tool enabling users of the SAFE Network to easily manage websites

[Shane]

I’ve been thinking about this since I woke up this morning, and I’ve come to the conclusion that I don’t and won’t have a MAID address for donations. I don’t like the idea of taking value out of the community when I haven’t provided any tangible value as of yet. Don’t get me wrong, it would be so easy to take money from everyone and parade around begging for donations to feed a lavish lifestyle, but that’s not me: I write code because I enjoy it, as soon as I take money it will begin to feel transactional.

In the future this stance might change, depending on which project I’m working on and what my financial situation is, but at least while I’m working on Phantom I have no intention of taking donations to support development. The only support I need is the of the community. That being said: thank you for the offer, it’s greatly appreciated.

[happybeing]

I understand this perspective @Shane and at the same time think it is ok to accept or ask for donations - people don’t have to donate. I think it’s important to be clear about the nature of any ‘transaction’ but it can be as simple as: “if you like what I do and want to show your appreciation through a donation I appreciate that very much, but accept on the understanding that it doesn’t come with any strings.”

My understanding of care work is that it is poorly paid, so it seems reasonable you might find a bit of extra income helpful.

Anyway, it’s great to see you building again and useful to be able learn from such an experienced developer.

[Shane]

Edit: Whoops, didn’t realise the weekly update would be at the same time, sorry for stealing some of your thunder!

Hey all, these updates seem to be coming thick and fast at the moment.

Editor

The editor now has support for the following markdown:

• *** horizontal rules
• ___ horizontal rules
• --- horizontal rules
• Inline code blocks using single backticks
• Multiline code spans using three backticks, as well as support for extracting the language you’re targeting.

You can see here the javascript language being extracted from the multiline code span as a CSS class you can target or extract to pass in to a syntax highlighter. There’s no support for any syntax highlighting out of the box due to the size it would add to the bundle, but with the upcoming theme system it will be very easy to add a syntax highlighter.

Theme system

I’ve been doing a lot of thinking and tinkering with the theme system over the past few days and I’ve settled on an idea I like. It’s worth mentioning: this is a first draft, work in progress, and is very likely to change based on feedback or code complexity.

For most users, they will simply go to the theme page, and either click on one they like, or import a theme from a SAFE network URL they found. This allows people to make and share themes easily, and for users to import an entire theme with just a single URL.

Theme manifests

The manifest describes the theme, gives it a displayable name, a banner image to show alongside the name, and tells the theme compiler about the assets the theme contains.

Theme templates

Themes are built using Vue.js, but don’t worry - I’ve abstracted away all of the complexity and left just a couple concerns to deal with. Theme template files contain the basic HTML for the webpage, as well as some text/x-template script tags which can be used for telling the Vue application what HTML it should use for each section.

… And that’s it. It’s that simple to create a theme. (obviously, you’d have to spend time styling it and what not, but that’s to be expected.) A theme author can create the manifest, publish the files which the manifest points to on the SAFE network and the Phantom theme compiler takes care of the rest. It inlines all of the assets and creates a hash-routing based single page Vue app.

Here is a work-in-progress example of a compiled website, ready to be deployed to the SAFE network:

At the moment, the only CSS in the theme is a red background, but all of the code is there and all I have left to do is create the “bundle” containing the information about the posts and generate a Vue single-page-app route for each post and it’s done.

The final size of a website deployed using Phantom is about 120KB (uncompressed!) and it contains all the code needed to bootstrap the application, handle routing and inject each post / generate a post listing page:

Like I’ve said, this is a working proof-of-concept and it’s a bit rough around the edges, but the long and short of it is:

• The Phantom app will contain a handful of basic themes
• End-users can select a theme by clicking a single button
• End-users can import a theme created by another author by clicking a button, inputting a URL and clicking “install”
• Theme authors can create and share themes easily and quickly, with next-to-zero knowledge of how Vue works internally.

All this functionality will be available within the SAFE network as soon as RFC0060 has been accepted / implemented, as without it there’s no way for web apps to create FilesContainers directly in the SAFE network. So at the moment, all of the SAFE calls are being mocked, but I’m fairly (99%) confident as soon as that RFC is implemented all I should need to do is deploy and it should work out of the box.

[Shane]

The last update was really rather developer focused, so I thought I would follow it up with a simple explanation for how themes work from a users’ perspective. You click the one you want and it automatically works. That’s it:

If you want to use a theme made by someone else (or yourself) you just click on Import theme and pop the SAFE URL in the modal which opens, and the theme appears in the list above.

My ultimate aim is:

• To make it as easy as possible for people to install themes, whilst:
• Making it as easy as possible to share themes with others, whilst:
• Making the process of creating a custom theme as simple as possible.

If I can hit all three of those, we are on to a winner in terms of user experience. I will, of course, be shipping with several themes, since they’re so quick to throw together.

[Nigel]

Huge props to your productivity and drive to make this so lightweight and inline with SAFE ideals. Great having you back Shane

[Shane]

Hey all, again, post updates seem to be coming thick and fast. I’ve really missed writing code, to be honest. I’m away this weekend for a party so I likely won’t be giving any updates again until Monday.

Deploying posts

On my post listing page, I can see two posts I’ve created previously, I select the top one for editing:

Using Markdown, I add a picture of a cat to the top of the post, then I save the post. It’s state is changed from Published to Draft. I can click on the Publish drafts button to deploy the posts and create a new NRS version:

Once I click it, it builds up the posts in to a bundle, as well as the Vue single page app needed for the posts to be displayed. It creates a bundle which is 124KB in size. This bundle contains every. single. line. of. code. required. It’s literally the only thing your users have to download, and as a single page app, they won’t trigger any more network requests.

At the moment, the SAFE network doesn’t GZIP content, but I want to stress that the 124KB is uncompressed. If the SAFE network ever adds GZIP download support, the bundle size will be reduced to around 70KB.

If I navigate to my NRS within the SAFE network, I’m now presented with my blog. It’s that easy. (The styling here is just for my testing, I’ll make the themes look a lot nicer than this):

If I click on the top post, It displays the post content:

Going forward

We’re now at a point where (barring the implementation of RFC0060 by Maidsafe for the API calls I need) we have a fully working app, which is very rough around the edges.

I still need to do the following:

• Add the option to add arbitrary pages, similar to how wordpress has “posts” and “pages”
• Add and style several basic themes
• Add some more error handling so it’s less fragile when things go wrong
• Add “custom fields” to themes, so theme publishers can offer configuration options to users
• Start to think about a plugin system, and how we can add dynamic content in the future (Comment systems, Ecommerce platforms, that kind of thing)

I’m going to take the weekend to think about how the custom fields will work, and how I want to integrate dynamic content.

[Southside]

Fantastic work, Shane.
Take the weekend off and go to a party

[stout77]

Enjoy the party Shane, amazing work!!

[Shane]

Hi all. Before I start I want to clarify that this update is geared towards future theme developers, as such, if you’re just going to be a normal user (someone who simply uses Phantom for writing and sharing blog posts), this probably won’t be relevant to you.

Theme.json and configuration

I’ve began work on adding the concept of “custom fields” to the theme.json files so that theme developers can add extra options for users to select from. I’m aiming for “powerful but easy”, so there will likely be some compromises here in the future, but it should GET THE JOB DONE.

Below you can see a (valid) theme.json file, from the Banana theme I’m using for testing:

It has the following REQUIRED data:

1. The name of the theme
2. A basic description to show the users on the themes page
3. A square banner image used to “fancy up” your theme on the themes listing page
4. A template which points to the basic HTML of your theme
5. An ordered array of scripts which will be imported in your theme.
6. An ordered array of stylesheets which will be imported in your theme.

It has the following OPTIONAL data:

• An array of configuration options you can display to the user.

If the config field is present, then the following data becomes REQUIRED:

• The name of the configuration option
• A basic description for the configuration option
• A count (containing either single or multi) which enables your users to input data multiple times if enabled (for instance, we have here a menu-items config where the user can input multiple menu items)
• An array of fields containing:
  • The name of the field
  • A basic description for the field
  • A HTML5 input type for the field (supported types are: text|email|date|number|tel|password|time|file)

Theme file linter

To make the work of creating / testing themes easier, I’ve created a simple linting tool which codifies these requirements.

For instance, if you try to import a theme.json file which has an invalid config field type, you will be presented with the following error:

Once your theme has been linted, it will be presented alongside the default themes, with the same “1 click install” as the other themes have:

Linting code

I don’t normally link directly to blocks of code, but I’m going to link the theme linter code right here

I’ve gone to the effort of being incredibly verbose and procedural when writing this, so yes, I’m aware it’s very repetitive, and I’m aware it could be simplified, but I figure that minimising the amount of magic happening here will make it as easy as possible for developers in the future to debug issues with their themes.

At the moment, the config fields don’t do anything - I need to add the Theme configuration page which will ingest them, spit out input fields, ingest the inputted data, and spit out something easy to query within the theme template. Expect to have an update with the code to manage that within the next few days.

[joshuef]

@Shane i’m not sure how you envisage adapting other themes. Wordpress has a concept of ‘child themes’, were you can only provide the parts you want to override.

There’s a benefit over just forking in that you’d get upstream changes automatically. Could work quite easily with xorurl of a ‘parent’ theme in the config perhaps. That’s just one idea anywho, wanted to throw it out there incase it impacts anything you’re on the now.

[Shane]

I like the idea and it wouldn’t be hard to compose something like that, let me have a tinker and get back to you with a working concept.

[Shane]

Thanks for the inheritance idea @joshuef.

Theme.json

I’ve added a parent field to the theme.json, so that themes can inherit config from parent themes. This enables theme authors to peg their theme to the default Vue themes, or even to themes created by someone else.

The themes are logically merged, with child values overriding parent values when it comes to the name, description, banner and template, and with a merge happening for styles, config and scripts - the merge is performed parent first, so that your styles and scripts can override and leverage the styles and scripts included by the parent theme.

In the themes which ship with Phantom, all the URLs are relative to Phantom, but these fields all obviously support any valid SAFE XorURL / NRS - relative or absolute.

Multiple inheritance

Theme manifests are merged recursively and support infinite parent themes. This means that a parent theme can have a parent theme of its own, allowing theme authors to really take advantage of this system. Here you can see a theme.json with a parent theme.json, which has its own parent test.json

This means that any previously REQUIRED field in the manifest is now OPTIONAL - so long as at least one of the parent themes in the parent chain defines that field.

Here you can see the final composition of the banana test theme, with the styles, scripts and config fully merged.

I’m going to need to have a think about how users will update their themes in future when new version are released.

I will probably do some sort of XorURL saving, where on the themes page end users will be able to click on a button and it will validate the XorURLs of each theme file to test if there are any updates.

[joshuef]

great to see this coming along man. exciting stuff!

[Shane]

Another update for you all. Updates probably seem smaller than they were last week. They aren’t - an awful lot of code is being rewritten to make things smoother or catch errors, etc. I don’t tend to talk about the banal work of bug fixing because there’s rarely anything to share (who wants an update full of Fixed a typo in a network error state when a theme fails to update? ), but a lot of time has been spent on that over the last few days.

Themes are now smooth AF.

I’ve essentially rewritten the theme code from the ground up now I know how I want it to work. Now, when you hit install on a theme it redeploys all assets (including posts) with the new theme configuration. Previously it wouldn’t update until you had edited a post. This just makes the theme system more unified and less confusing to use.

Logout

The keen of eye might notice there’s now a logout button in the sidebar which wasn’t previously there. Until now, the default state of the app has been authenticated, now it defaults to unauthenticated, and…

An actual freaking website

Users who aren’t (or fail to) authenticate will no longer be shown the word “Dashboard” in a 70px red font. There’s now an actual website with some basic content.

My plan for the Built with Phantom section is to showcase cards linking to blogs which have been made using the platform. I’ll likely rotate these on a weekly-or-so basis. I want to stress now that these will be user submitted, not collected by the app. Phantom does not and will never send statistics about usage or any kind of telemetry back to me.

I have a few things left to do before the project is complete:

1. I need to add the creation of arbitrary pages. This is actually very easy (I’ve done it before with posts), but I’m suffering from Second Sock Syndrome
2. I need to add the page which displays custom theme configuration to end users. Again, not difficult, it’s just on the list of things.
3. Add the plugin / module system to support future work.
4. We need the implementation of RFC0060 which is sort of stuck waiting for acceptance from the Maidsafe team.

We’re probably looking at between a week and 10 days to get all this done and bugfixed. I’ll then probably spend 2-3 days just churning out simple themes until we have around 10.

Yozu software meetup

My old protege, @AndyAlban’s company Yozu are hosting a software developer’s meetup in Liverpool tonight for anyone interested. I’ll be there as well and if any of you turn up I’d love to chat SAFE, VueJS or anything tech related really. They offer free pizza and free drinks, although I mainly go for a chance to catch up with Andy. I have no relation to Yozu and haven’t been asked or paid in any way to advertise the meetup, I just thought it would be a cool chance to meet some of the forum members if Liverpool is convenient and the late hour isn’t too inconvenient.

[anon78865233]

Wow, you really know your stuff. An asset to this community and network

[Shane]

Another update? Yeah, another update!

Theme configuration

For context going forwards, here is the configuration for the Light theme I’m using for testing. The only parts relevant to this update post are the config fields. It’s reasonably easy to follow even if you don’t code:

• We define a single entry field for a logo in the header
• We define a multi entry field for menu items in the header.

Which leads us on to:

Theme configuration page

If your theme ships with configuration (it’s optional, and I imagine some themes will want to be “works out of the box simple” and opt-out of any config fields, and the user installs your theme, then a Theme config link will appear in the sidebar. If you click on that link, you will see this page:

This is the page with 0 configuration fields entered. single entry fields show the inputs by default. multi entry fields default to showing nothing but an Add button. Multi entry fields support theoretically infinite values (the actual hard limit is somewhere around 5MB of raw data, which should be plenty).

Here is the page with some user generate content:

Configuration is edited on a per theme & per blog basis, so you can use the same theme on two different websites you deploy with Phantom but have different theme configuration entirely. This just makes it as flexible as possible.

All of this is coded and working. I need to add support for an image upload field, but apart from that we’re good. (Don’t worry, images won’t count towards the 5MB storage limit, the images will be uploaded automatically and just the XorURL of the image will have to be stored). All I need to do now is work out how I want to expose this data to the theme once it’s compiled. The lazy way would just be to encode the field data as JSON and spit it out in to a global (thereby letting theme authors worry about how to ingest it), and absent of any amazing ideas that’s likely what I’ll do.

Bundle size

Here you can see the compiled size of the phantom website itself, not a blog generated by us (which are still around 120KB in total). I’ve spoke before about wanting to make it as small and as fast as possible. At the moment, Phantom and all its dependencies is around 200KB uncompressed, with 3 images (the image on the home page, and the banner images for each theme) an 2 fonts bringing the total size of the application up to around 350KB. Compressed, this could be as low as 190KB. Given that according to HTTPArchive the average website weighs around 3MB, I’m pretty happy to have produced something 1/10th of the size.

(In the above image, I’ve hidden the dependencies of the compiled blog websites, since they don’t factor in to the Phantom application)

Work left to do is as follows:

• Allow creation of arbitrary pages
• Easy image uploader for posts / pages / theme configuration
• Create the module system
• A bunch of bugfixing

[Shane]

Theme updating

One of the key problems with the theme system (from the perspective of a theme author) has been that it’s static. Sure, you have inheritance but once it’s installed it’s stored in memory forever. Over the past 24 hours I’ve been working on making the themes update-able.

Theme config example

Here you can see the theme config page for the Light theme, with a single option: menu-items.

We go to the theme code and update it, adding a new logo field:

I want to stress that I’m making these changes locally, but all of this stuff will work over the network. As long as you publish your theme to the same XorURL, autoupdating will work.

We can then go to the themes page and click on the Would you like to check for updates? button. In the future I’ll probably do some fancy XorURL checking, but this will do for now.

When we go back to the Light theme config page, we can see that the logo field is now being presented to the user:

I’m going to be working on getting that file input working today, so that we can add the image uploader to the posts page. We’re very close to finishing this project now (at least for a 0.0.1 release).

[dirvine]

Man @Shane you are a machine, it’s massively einvigourating to see this. Thanks from all of us.

[happybeing]

Yes it’s a joy to watch. I always hoped @Shane would return because he brought so much in a short time. Great to have you back man.

[Shane]

File uploading

I’ve spent the last day working on making the uploading of files as easy as possible. At the moment, a lot of the tooling built for the SAFE network is “by developers, for developers” and I want to move away from that and build tools which every-day-normal-people will be able to use intuitively. Intuition is so important when you’re trying to convince people to make a huge change.

Here you can see the theme config page, with a file input for a logo in the header:

When you click on the Choose file button you see the standard Explorer window:

When you select a file it is instantly uploaded and an XorURL is displayed (worth mentioning the XorURL you see here was generated by my local mocking library and it as such won’t work if you put it in to a SAFE browser):

I need to complete the work of pulling the theme config in to the compiled websites, but that won’t take me very long now I’ve got this all working (It’s basically just spitting out a JSON string). We’re at a point now where the theme system is complete.

When you upload a file, the theme authors will have access to two values, the XorURL of the uploaded file and the inferred mime type:

As always with inferred mime types, the user lies, the computer lies, the implementation is incorrect and as such it’s not worth trusting except for deciding on how to display the content.

Left to do:

• Pull the theme config in to the compiled websites this is now done
• Get the image uploader working on posts and pages
• Get arbitrary pages working
• We’re still waiting for a response from Maidsafe on RFC0060

At that point the core of Phantom will be considered complete and I’ll push a 0.0.1 release while I begin public work on the plugins / modules code.