Widgets feature

[Frederico Caldeira Knabben]

Widgets are represented by a group of elements that act as a single entity inside the editable content. The widget elements are many times protected from direct editing and can be configured to have different behavior, features or layout.

A good example for a widgets is a "captioned image", which contains an image element and text for the caption. Both parts will always live together and should be selected as a whole. Once deleted, both should be removed.

[Frederico Caldeira Knabben]

Pushed the first prototype to ​t/9764@cksource.

The current version is still very limited. It is based on our research to understand the feasibility of such feature and also to propose a few UX models to be discussed. It is mainly a feedback ready version. Implemented features have been crossed out in the ticket description.

So far, excluding the non implemented restrictions, it works well with Chrome... almost well with Firefox... maybe ok with Opera... WTF with IE9. So, test it with Chrome for now.

[Dinu]

This is huge! I'm a kid who got his first bycicle :)

I was thinking about the copy&paste, drag&drop features and the following came to my mind: There can be stateful widgets that don't like to be dragged across editors. Presumably in the end the widgets will be implemented as plugins, that will generate, be able to edit & shape non-editable content via a properties dialog etc. Since different editors may have different lists of plugins loaded by configuration, it would make no sense moving a widget to an editor that does not support it. So, to further the already ambitious checklist, I think such a mechanism should be implemented that:

• a widget can have a bit of metadata, say "widgetClass" that says the widget is an instance of a "captionedPicture"
• a hookable API callback (or some other form of appropriate event handling) that can process any cross-editor operation, such as safely serializing the widget for transport (copy), safely deserializing the widget (paste)

I don't know how to formulate this in a single sentence that can be added to the checklist :)

[Frederico Caldeira Knabben]

Replying to dinu:

Interesting aspect. Let's see what can be done in that sense once we have the copy/paste and DnD thing researched. I've added a notice in the list.

[Dinu]

Replying to fredck:

Indeed... there's time for that. But since my mind is still somewhat on that: since I mentioned it, the idea of custom serialization/deserialization of the widgets has really caught my imagination, because it would make this dream possible: posting back and forth the server a piece of xml that could look like:

Blahblah
<foo:captionedPicture 
  pictureId="id_set_via_an_ajaxed_property_dialog"
  url="url_for_said_id">
        <foo:caption>
            The user<b>edited</b> me
        </foo:caption>
</foo:captionedPicture>

Where this is going is pretty obvious... "Paste from MS-Word? Na... I got me own CK-Word!"

[Frederico Caldeira Knabben]

Replying to dinu:

At least for widget data embedded into the document contents, we're opting for a much simpler solution, based on HTML5+RDFa. I mean, that's the format the widgets system will look for widgets data. This is in fact data that the browser understands, when it comes to rendering, and that can hold metadata.

Then, each plugin will have total freedom to use the data format that fits best the developer needs, transforming to and from HTML5+RDFa on input and output. Each one will be able to decide whether to do it on client side (plugin code) or server side.

[Frederico Caldeira Knabben]

A discussion about the Properties Panel has been opened at our dev mailing list:
​https://groups.google.com/d/topic/ckeditor-dev/uHHicc9J11A/discussion

[Dinu]

Replying to fredck:

Makes sense when you put it this way :) So, use the same system for copy/paste? (ask plugin to transform to canonical form before copy, and reverse for paste)

[Wim Leers]

I posted feedback regarding CKEditor Widgets over at ​http://drupal.org/node/1260052#comment-6836034.

Here's the portion of that feedback relevant to this ticket:

• TRIVIAL: no events yet to know that the "CKEditor Widgets content" has changed; a specific event is not necessary, as long as the same "changed" event is triggered that gets triggered whenever you type something new in the "regular CKEditor" is also triggered when "CKEditor Widgets content" changes, then it's fine.
• EASY: CKEditor Widgets' HTML/DOM representation does not yet get "cleaned up" when saving: we don't want a bunch of hardcoded HTML to show up — somebody in the 9764 ticket calls this "(de)serialization" which is another way of looking at it. We e.g. don't want an <iframe> with YouTube stuff to be saved; instead we want something like <youtube data-video-id="">; similarly we don't want <figure class="caption" about="something"><img src="" data-cke-saved-src="" /><figcaption>blah</figcaption</figure>, instead we want something like <img src="" data-caption="" data-align="" />. That means we have to implement both a server-side and a client-side "render" or "theme" function that maps from the canonical representation to whatever way a specific site wants it to look. This is also the way we made it work in Aloha.
  In short: Drupal themers need to be able to define what the HTML should look like. Existing, "minimal/core" HTML elements must be annotated with data- attributes, which also simplifies data migration: it needs to be possible to stop using a Drupal filter and have the image captions disappear, i.e. simply fall back to regular images (<img />). Analogously, it needs to be possible for us to import data from legacy systems where there are just images (<img />) and have CKEditor annotate them with data- attributes to add captions to them.
  If we wouldn't have this ability, then we'd be saving a lot of crufty/very unstructured data into text/markup stored in Drupal, and that's something that is considered to be utterly bad and evil in the Drupal world.
  Let me know if this is not yet sufficiently clear.
• HARD: ability to convert an image (<img />) to a captioned image (<img data-caption="" />). But as said above, it must still be possible to render it (which in Drupal would happen through Drupal's theme layer) to something else than that entirely.

Thoughts? :)

[Frederico Caldeira Knabben]

Replying to dinu:

Makes sense when you put it this way :) So, use the same system for copy/paste? (ask plugin to transform to canonical form before copy, and reverse for paste)

No, I don't think this will be needed.

Copy and paste within the editor should work without any intervention. This is our goal at least.

Copy and paste outside the editor... much probably the user is interested on the rendered version of the widget, not its canonical form. This means that we'll not have to deal with transformations, leaving the HTML5+RDFa hit the clipboard.

[Frederico Caldeira Knabben]

Replying to Wim Leers:

• TRIVIAL: no events yet to know that the "CKEditor Widgets content" has changed; a specific event is not necessary, as long as the same "changed" event is triggered that gets triggered whenever you type something new in the "regular CKEditor" is also triggered when "CKEditor Widgets content" changes, then it's fine.

I believe you're talking about the so called "onchange" event, right? If yes, that's another discussion topic and there are possible solutions for it.

// Anyway, it's not the first time we've been hit by this need, so we may think about something on core. Actually, I've always though about using the undo system for this, which may be more reliable.

[Frederico Caldeira Knabben]

Replying to Wim Leers:

• EASY: CKEditor Widgets' HTML/DOM representation does not yet get "cleaned up" when saving: we don't want a bunch of hardcoded HTML to show up — somebody in the 9764 ticket calls this "(de)serialization" which is another way of looking at it. We e.g. don't want an <iframe> with YouTube stuff to be saved; instead we want something like <youtube data-video-id="">; similarly we don't want <figure class="caption" about="something"><img src="" data-cke-saved-src="" /><figcaption>blah</figcaption</figure>, instead we want something like <img src="" data-caption="" data-align="" />. That means we have to implement both a server-side and a client-side "render" or "theme" function that maps from the canonical representation to whatever way a specific site wants it to look. This is also the way we made it work in Aloha.
  In short: Drupal themers need to be able to define what the HTML should look like. Existing, "minimal/core" HTML elements must be annotated with data- attributes, which also simplifies data migration: it needs to be possible to stop using a Drupal filter and have the image captions disappear, i.e. simply fall back to regular images (<img />). Analogously, it needs to be possible for us to import data from legacy systems where there are just images (<img />) and have CKEditor annotate them with data- attributes to add captions to them.
  If we wouldn't have this ability, then we'd be saving a lot of crufty/very unstructured data into text/markup stored in Drupal, and that's something that is considered to be utterly bad and evil in the Drupal world.
  Let me know if this is not yet sufficiently clear.

In the very beginning of the prototype thoughts, we talked about the data input/output. In fact, we checked ​Drupal's Caption Filter module, which handles this kind of snippet:

[caption]<img src="" alt="" />This is an image caption[/caption]

At first we said, "ok, let's make it work with that data format". This is totally doable with CKEditor.

Later we came to another conclusion, which reflects your words. If we have a modular server side system (like Drupal), it is hard to know precisely what data format will be there. At that point we decided that the widget system (read the "widget" plugin) will work under a common standard format: HTML+RDFa.

On top of that, we started developing the prototype widgets to check if the idea worked. At that point, we were not any more dealing with data transformations, assuming that HTML+RDFa will arrive from the server.

But, this assumption is not a rule. It was just a prototype simplification. We could have each specific plugin certainly touching the input data, transforming the canonical data into HTML+RDFa before having the widget plugin doing its job. So, at the current stage, data transformation at the client side is possible. We're simply not doing it in this prototype.

Still, something came to my mind to say that client side manipulation may be wrong (in Drupal's case). Our intention is bringing the WYSIWYG feeling to the user. Due to the extreme flexibility of Drupal (modules, filters, hooks, etc.), we could have more than one part of the code touching the final rendered version of widgets. So it would be just a matter of properly introducing RDFa annotation to the touched HTML during this process. Then, the opposite server side task is done when receiving the data output. Doing the canonical->rendering format transformation in the client will just add a unnecessary level of complexity, which is not even the usual way of doing Drupal stuff.

Anyway, my talk got a bit of topic. To summarize, it is still possible to do the canonical data transformation in the client side. It's up to the developer to decide whether this is the right way for it or not.

In any case I think it would be worth enlarging the prototype to have a single image caption plugin handling different annotated HTML representations, for example <img src="">, <img src="" data-caption="" data-align=""> and the current <figure>...</figure>.

[Wim Leers]

Replying to fredck:

Replying to Wim Leers:

• TRIVIAL: no events yet to know that the "CKEditor Widgets content" has changed; a specific event is not necessary, as long as the same "changed" event is triggered that gets triggered whenever you type something new in the "regular CKEditor" is also triggered when "CKEditor Widgets content" changes, then it's fine.

I believe you're talking about the so called "onchange" event, right? If yes, that's another discussion topic and there are possible solutions for it.

// Anyway, it's not the first time we've been hit by this need, so we may think about something on core. Actually, I've always though about using the undo system for this, which may be more reliable.

Exactly. This is something we need from CKEditor in general for Drupal, too, btw. See ​http://drupal.org/node/1260052#comment-6841372, gap number 3 :)

[Frederico Caldeira Knabben]

Replying to Wim Leers:

• HARD: ability to convert an image (<img />) to a captioned image (<img data-caption="" />). But as said above, it must still be possible to render it (which in Drupal would happen through Drupal's theme layer) to something else than that entirely.

Do you mean that it may happen that <img /> may have a totally different rendering than <img data-caption="" />? If that is true, it means that we may have still other filters adding "data-" stuff to <img>, which may eventually impact further on the rendering.

At that point, changes to widget properties in the editor may result on different rendering strategies, which the editor should dynamically refresh.

If all this is true, there are two options:

1. Ajax call to the server on properties panel "Ok", so the widget is rendered again in the server.
2. Have rendering functions in the client side, which would be duplications of the functions present on the server side.

I know it may sound strange, but I would go with "1". The reason is partially what I said on comment:15 and also the fact that maintaining duplicated code is not the best think one could propose around.

I need some inputs on this, just to be sure I'm not overcomplicating it.

[Wim Leers]

Replying to fredck:

Replying to Wim Leers:

• EASY: CKEditor Widgets' HTML/DOM representation does not yet get "cleaned up" when saving: we don't want a bunch of hardcoded HTML to show up — somebody in the 9764 ticket calls this "(de)serialization" which is another way of looking at it. We e.g. don't want an <iframe> with YouTube stuff to be saved; instead we want something like <youtube data-video-id="">; similarly we don't want <figure class="caption" about="something"><img src="" data-cke-saved-src="" /><figcaption>blah</figcaption</figure>, instead we want something like <img src="" data-caption="" data-align="" />. That means we have to implement both a server-side and a client-side "render" or "theme" function that maps from the canonical representation to whatever way a specific site wants it to look. This is also the way we made it work in Aloha.
  In short: Drupal themers need to be able to define what the HTML should look like. Existing, "minimal/core" HTML elements must be annotated with data- attributes, which also simplifies data migration: it needs to be possible to stop using a Drupal filter and have the image captions disappear, i.e. simply fall back to regular images (<img />). Analogously, it needs to be possible for us to import data from legacy systems where there are just images (<img />) and have CKEditor annotate them with data- attributes to add captions to them.
  If we wouldn't have this ability, then we'd be saving a lot of crufty/very unstructured data into text/markup stored in Drupal, and that's something that is considered to be utterly bad and evil in the Drupal world.
  Let me know if this is not yet sufficiently clear.

In the very beginning of the prototype thoughts, we talked about the data input/output. In fact, we checked ​Drupal's Caption Filter module, which handles this kind of snippet:

[caption]<img src="" alt="" />This is an image caption[/caption]

At first we said, "ok, let's make it work with that data format". This is totally doable with CKEditor.

Later we came to another conclusion, which reflects your words. If we have a modular server side system (like Drupal), it is hard to know precisely what data format will be there. At that point we decided that the widget system (read the "widget" plugin) will work under a common standard format: HTML+RDFa.

On top of that, we started developing the prototype widgets to check if the idea worked. At that point, we were not any more dealing with data transformations, assuming that HTML+RDFa will arrive from the server.

But, this assumption is not a rule. It was just a prototype simplification. We could have each specific plugin certainly touching the input data, transforming the canonical data into HTML+RDFa before having the widget plugin doing its job. So, at the current stage, data transformation at the client side is possible. We're simply not doing it in this prototype.

Excellent!

So… I think we're saying the same thing, but I'm not entirely sure :)

I'll try to clarify what we need to be able to build/configure for this to work with Drupal's filter system.

• The content in the DB looks like this:

  <h1>Hello, world!</h1>
  <img src="earth.jpg" alt="Photograph of planet Earth." title="Planet Earth" data-caption="Planet Earth" data-align="left" />
  <p>Our planet is called <strike>the planet</strike> Earth.</p>
  

• Upon rendering a page with this content, the above content goes through Drupal's filter system (in PHP). There are two filters configured, and they run in a specific order:
  • First: a filter that limits the allowed HTML tags. This is a filter of the FILTER_TYPE_HTML_RESTRICTOR type. The h1, img and p tags are allowed, but the strike tag is disallowed. When this filter is ready, the content looks like this:

    <h1>Hello, world!</h1>
    <img src="earth.jpg" alt="Photograph of planet Earth." title="Planet Earth" data-caption="Planet Earth" data-align="left" />
    <p>Our planet is called the planet Earth.</p>
    

  • Second: a filter that is similar to the Caption Filter module you link to above, but that doesn't use BBCode-like syntax, but adds captions based on data- attributes. This is a filter of the FILTER_TYPE_TRANSFORM_REVERSIBLE type (i.e. reversible transformations; examples of irreversible transformations are e.g. automatically linkifying keywords and replacing "typical keyboard quotes" with "typographically correct quotes"). This filter employes DOM parsing (in PHP) to find img tags that have either the data-caption or the data-align attributes set. If either is present, it will transform the original img tag's HTML into something else. Drupal themers can override what that will look like by implementing the ​`theme_caption()` function (again, in PHP). The default theme_caption() function will cause the content received to be transformed further to

    <h1>Hello, world!</h1>
    <div class="caption caption-left">
      <div class="caption-inner" style="width: 100px;">
        <img src="earth.jpg" alt="Photograph of planet Earth." title="Planet Earth" />
        <div class="caption-text">Planet Earth</div>
      </div>
    </div>
    <p>Our planet is called the planet Earth.</p>
    

• Until now, no CKEditor, and no JS was involved at all. This is a simplified version of the render pipeline. Note how the rendered version is something that you absolutely don't want to be stored in your DB, because it can be "themed" (rendered) in all sorts of crazy ways.
• Now imagine you're going to do in-place editing in Drupal. Then we'd use CKEditor 4's inline editing ability. But clearly, feeding it this sort of mark-up is of course crazy; we don't want users to edit that div hell!
  So, what Drupal's in-place editing functionality will do is: automatically detect that "transformation filters" have been applied, going back to the server and asking for a version without the transformations. We then replace what was originally on the page with that version and then call CKEDITOR.replace() on it (or whatever the equivalent is for CKEditor's inline editing mode.
  (Note that on the Drupal back-end, where everything is form-driven you'd simply get the content stored in the DB in a textarea, call CKEditor.replace() on it and then have a semantically/functionally identical case.)
• Now it becomes interesting. To achieve "true WYSIWYG", it is necessary for our content's img tag with data-caption and data-align attributes to be rendered in an identical way.
  What we did with Aloha Blocks, was this:
  • Ensure the "Aloha Block type" (in this case, for captioned images) has an optional render callback (so Aloha could ship with a default) that gets called in its ​initialization phase. "CKEditor Widgets" would need something similar.
  • An implementation of the aforementioned theme_caption() (which was in PHP), but then in JS: ​`Drupal.theme.caption()`. The JS code is truly identical to the PHP code, bar syntax differences.
  • And finally: have an ​adapter function that maps between how Drupal JS theme functions work and how the Aloha Block's code calls its.
• However, we still don't want this div hell to be stored in our DB: we need to ensure that upon saving, we clean up after ourselves. We must be able to replace the rendered version (as rendered by Drupal.theme.caption in this example) with the original mark-up, but with the updated data- attributes.

I hope the above walkthrough makes sense to you.

Still, something came to my mind to say that client side manipulation may be wrong (in Drupal's case). Our intention is bringing the WYSIWYG feeling to the user. Due to the extreme flexibility of Drupal (modules, filters, hooks, etc.), we could have more than one part of the code touching the final rendered version of widgets. So it would be just a matter of properly introducing RDFa annotation to the touched HTML during this process. Then, the opposite server side task is done when receiving the data output. Doing the canonical->rendering format transformation in the client will just add a unnecessary level of complexity, which is not even the usual way of doing Drupal stuff.

This is where I have to disagree.

I do see your point of having "more than one part of the code touching the final rendered version of the widgets". However, I can guarantee you that that is (fortunately!) not the case. Maybe in the far future, but for now this sort of thing always maps to a single Drupal "theme" function. Otherwise, we'd all be going insane :)

This particular part is of great interest to me:

So it would be just a matter of properly introducing RDFa annotation to the touched HTML during this process. Then, the opposite server side task is done when receiving the data output.

If I understand you correctly, you're saying that the eventual CKEditor Widget's HTML, as changed by potentially many Drupal "theme" functions (in JS) would be annotated with more RDFa (for e.g. the caption or alignment), and it would be up to the server-side (PHP) code to strip away all of the "presentational HTML" and reduce/transform the HTML back to its original canonical form? This is something that is absolutely a no-go for Drupal.

And this particular part too:

Doing the canonical->rendering format transformation in the client will just add a unnecessary level of complexity, which is not even the usual way of doing Drupal stuff.

I don't see at all why this is an unnecessary level of complexity? This is specifically tying the complexity of dealing with this to the client side. The server-side should only have to deal with a one-way transformation: for displaying the page. If I feed the client-side <img src="foo.jpg" data-caption="bar" /> and change the caption to baz, then I want the client side to save <img src="foo.jpg" data-caption="baz" />, and not shift the burden of dealing with a potential div hell to the server side.

Anyway, my talk got a bit of topic. To summarize, it is still possible to do the canonical data transformation in the client side. It's up to the developer to decide whether this is the right way for it or not.

We need a consistent render infrastructure to be an inherent part of CKEditor Widgets' API.

In any case I think it would be worth enlarging the prototype to have a single image caption plugin handling different annotated HTML representations, for example <img src="">, <img src="" data-caption="" data-align=""> and the current <figure>...</figure>.

Agreed!

[Wim Leers]

Replying to fredck:

Replying to Wim Leers:

• HARD: ability to convert an image (<img />) to a captioned image (<img data-caption="" />). But as said above, it must still be possible to render it (which in Drupal would happen through Drupal's theme layer) to something else than that entirely.

Do you mean that it may happen that <img /> may have a totally different rendering than <img data-caption="" />? If that is true, it means that we may have still other filters adding "data-" stuff to <img>, which may eventually impact further on the rendering.

At that point, changes to widget properties in the editor may result on different rendering strategies, which the editor should dynamically refresh.

If all this is true, there are two options:

1. Ajax call to the server on properties panel "Ok", so the widget is rendered again in the server.
2. Have rendering functions in the client side, which would be duplications of the functions present on the server side.

I know it may sound strange, but I would go with "1". The reason is partially what I said on comment:15 and also the fact that maintaining duplicated code is not the best think one could propose around.

I need some inputs on this, just to be sure I'm not overcomplicating it.

If I were to consider the two options you propose, I'd always choose option 2. And I need only a single reason: performance. Going back and forth to the server (and the inherent delays due to latency) will ruin the "true WYSIWYG" experience, since it's not instantaneous anymore.

---

You make an interesting point regarding other CKEditor widgets or Drupal filters adding more data- attributes. However, that quickly leads to very, very nasty & complex scenarios :) I think it's a fair assumption to say that each HTML element can only be one CKEditor Widget at a time.

I can understand your reasoning, but IMHO the one single design choice/assumption that you made in comment:15 that has overcomplicated things, is where you say "more than one part of the code touching the final rendered version of the widgets". If you instead assume a single function/object/whatever being responsible for the rendering, then all that complexity goes away :)

[Dinu]

I'd like to come in the middle here...

1) For me, Fred is making perfect sense the best way to go is to have widgets re-rendered server-side. What JS can mirror of Drupal (or any other CMS) without any AJAX calls is semantically small-talk. For instance. To change the image in the image-with-caption example you will need to go to the server, either to browse available images or upload a new one. Ideally for any CMS, it will get a unique ID, and a storage context. This requires a request to the server. The server can, on this request, post back the updated form of the widget. So, in saying "we don't want anything posted back to the server for rendering the widget", you are saying "we'll make a widget that only works with one picture: the initial one". I do realize other widget parameters may work extremely well on a short JS-to-HTML cycle... but to me, the one-size-fits-all model seems to be the one Fred proposed.

2) But, precisely because of 1), I think one needs full control in case of content duplication (copy-paste, drag-n-drop, etc). While the layout may be good in the editor, back on the server it will be a nightmare to know who-did-what. Simple example: say my picture-with-caption widgets are for some reason stored in a separate table in the server, with a unique id (so that, for instance, I can know what descriptions users added for a particular image), and in the stored html snippet the widget is trimmed to something like <foo:pictureWithCaption id="123" />. Now when I copy this widget to a new editor, its unique id will get copied too. This will mean, when I change its caption in one editor, it will change in the other, too. Not a good outcome. I want to be able to give it a new id on copy (actually, rather mark it was pasted and a new id will be provided for it on the server) or deny the copy altogether. Second nightmare scenario: pasting a widget inside another (or copying it inside itself?). This is a simple example, but there can be so many more ways state can be broken between the server and frontend by moving content between editors. And since this is best handled through #9829, it will make also 1) possible in the way Wim described it, making everyone happy.

3) "If you instead assume a single function/object/whatever being responsible for the rendering, then all that complexity goes away". With me it's a bit of an understatement. If you allow for nested widgets. While you have one widget per DOM node, the deserialization scheme will be nothing simple. Good planning can't hurt if you are ever to support such feature.

[Wim Leers]

1) For me, Fred is making perfect sense the best way to go is to have widgets re-rendered server-side. What JS can mirror of Drupal (or any other CMS) without any AJAX calls is semantically small-talk.

Please improve this sentence because it is hard to fully understand.

What is clear though, is that you're in favor of "let the server render a widget after every change of a property". I cannot stress enough that this is an absolute nightmare for performance. Please see comment:19.

For instance. To change the image in the image-with-caption example you will need to go to the server, either to browse available images or upload a new one.

Fair enough. To upload a new image, you obviously need to interact with the server. To select an image from an image gallery, you also need to interact with the server in the sense that the server has to serve the image gallery. But in either case only one piece of data is needed from the server: an image URL (and potentially width/height).

In the case of the image gallery, the server has generated the gallery, but the JS should be able to immediately (without going back to the server), if not, then it's a very weird image gallery.

Most importantly, the changing/selecting/adding of an image is just one of the potential actions for a "caption-with-image" Widget. The other actions are: change the caption text and change the alignment. If I understand you correctly, it will be necessary for the server to re-render the "caption-with-image" Widget on the server. Surely this is not what you intend? Imagine changing the "alignment" property and having to wait >2 seconds for this to take place… that would be horrible UX.

Ideally for any CMS, it will get a unique ID, and a storage context. This requires a request to the server. The server can, on this request, post back the updated form of the widget. So, in saying "we don't want anything posted back to the server for rendering the widget", you are saying "we'll make a widget that only works with one picture: the initial one". I do realize other widget parameters may work extremely well on a short JS-to-HTML cycle... but to me, the one-size-fits-all model seems to be the one Fred proposed.

That's not at all what I said! Changing the image implies changing the src attribute (and the widthand height attributes too). Of course that must be supported. RE: "a unique ID and storage context": in Drupal, each uploaded file gets a unique ID indeed (but if the same file is used multiple times, it's still referred to by a single ID). By "storage context" I think you actually mean "usage context", i.e. in which piece of HTML or which "field" of a piece of content this file (with a unique ID) is used, and this, too, is supported by Drupal: a DB file_usage table is maintained in which (file ID, content ID, count) tuples are stored. However, this table is updated when saving content, not when editing content.

(Note that in the case of Drupal, "fields" that are in fact just a piece of text, in this case HTML — it could also be Markdown or whatnot — are never parsed further out into more structured data. Drupal's "fields" are the atomic building blocks. So a "body" field that contains the body of an 2,000 word article does not get split into more things, nor do things get extracted from it. Instead, what would happen, is that you have e.g. "image" fields that are hidden from the end user, but are referenced from the body. Drupal builds and assembles HTML from a data structure, it doesn't disassemble HTML into a data structure. I think most CMSes work like this, but I of course may be wrong :).
I think this may part of why you think it's necessary to always talk to the server; if structured data is derived from the HTML you write with CKEditor, then I understand your reasoning, but in the case of Drupal, this would never happen; it'd be the other way around.)

2) But, precisely because of 1), I think one needs full control in case of content duplication (copy-paste, drag-n-drop, etc). While the layout may be good in the editor, back on the server it will be a nightmare to know who-did-what. Simple example: say my picture-with-caption widgets are for some reason stored in a separate table in the server, with a unique id (so that, for instance, I can know what descriptions users added for a particular image), and in the stored html snippet the widget is trimmed to something like <foo:pictureWithCaption id="123" />. Now when I copy this widget to a new editor, its unique id will get copied too. This will mean, when I change its caption in one editor, it will change in the other, too. Not a good outcome. I want to be able to give it a new id on copy (actually, rather mark it was pasted and a new id will be provided for it on the server) or deny the copy altogether. Second nightmare scenario: pasting a widget inside another (or copying it inside itself?). This is a simple example, but there can be so many more ways state can be broken between the server and frontend by moving content between editors. And since this is best handled through #9829, it will make also 1) possible in the way Wim described it, making everyone happy.

Copy/pasting Widgets we did not even consider (yet). I'm not sure how you'd pull that off. After all, the Widget is rendered in a certain way, and I don't think it's possible to intercept the "copy" event so that you would be able to pass it structured data instead of the exact HTML? Nested Widgets sound pretty insane and I think we can only discuss that with a reasonable confidence level once the non-nested version is up & running.

3) "If you instead assume a single function/object/whatever being responsible for the rendering, then all that complexity goes away". With me it's a bit of an understatement. If you allow for nested widgets. While you have one widget per DOM node, the deserialization scheme will be nothing simple. Good planning can't hurt if you are ever to support such feature.

This paragraph could also be clarified by cleaning up the language. It's very hard to understand what you mean exactly :(

I agree that thinking the Widget feature through is very important. But can you really think of a case where nested Widgets are a vitally important feature? Can you think of a way to implement nested Widgets in a way that will result in reliable, sane code? I can't see that just yet.

IMHO it makes more sense to implement an easy-to-use, easy-to-implement Widgets API at this stage, and if the need arises, implement a more advanced version in the future. Simplicity is important if we want this to take off and if we want people to build many Widgets and experiment with it.

[Michael Camden]

Would this feature also support the Japanese Ruby character? ​http://en.wikipedia.org/wiki/Ruby_character#HTML_markup

Here is a simple example;

<ruby>
<rb>東</rb><rp>(</rp><rt>とう</rt><rp>)</rp>
<rb>京</rb><rp>(</rp><rt>きょう</rt><rp>)</rp>
</ruby>

[Dinu]

Replying to Wim Leers:

1) For me, Fred is making perfect sense the best way to go is to have widgets re-rendered server-side. What JS can mirror of Drupal (or any other CMS) without any AJAX calls is semantically small-talk.

Please improve this sentence because it is hard to fully understand.

Well, the whole point of having the function of your CMS behind (with templates, filters, what-not), is to generate semantically difficult content, using many resources stored and cross-referenced on the server. I don't see how one could mirror that in javascript. I tend to believe this was also what Fred meant by "Due to the extreme flexibility of Drupal (modules, filters, hooks, etc.), we could have more than one part of the code touching the final rendered version of widgets". It's all a matter of how you percieve these widgets. I see them taking every bit of your CMS into the editor, giving you the possibility to include your site into itself, create a time paradox and be killed by your twin self from the future. Or just have the features MS-Word has and Google Docs are trying to build. I don't think we are barking at the wrong tree here, this discussion is (for me) what sets CKEditor apart from others. I take it from your view that you see the widgets as something-like-but-not-quite configurable HTML templates. I say "not quite" because you do want semantic CMS data produced, not just plain HTML. Functionality and usability are often seen throwing punches at each other, I guess.

What is clear though, is that you're in favor of "let the server render a widget after every change of a property". I cannot stress enough that this is an absolute nightmare for performance. Please see comment:19.

Come on, it's not a nightmare, it's a dream. Honestly. It's a feature that noone has implemented yet. So it's an absolute v1, right? I don't see how something that is an absolute first could be a nightmare for anyone. Except increased government taxes. And a huge meteor.

In the case of the image gallery, the server has generated the gallery, but the JS should be able to immediately (without going back to the server), if not, then it's a very weird image gallery.

It doesn't need to go back to the server. The server can send back the url and the whole HTML widget code.

The other actions are: change the caption text and change the alignment.

If you change the alignment property through the properties box, that can also set a data- attribute on the widget node when clicking ok... so no request to the server is necessary.

Or what you need to achieve is transform a <span style="text-align:right"> into a data- property?

Imagine changing the "alignment" property and having to wait >2 seconds for this to take place… that would be horrible UX.

If the roundtrip is 2s, the user will have a lot more to complain on than the latest feature of the editor, I think...? Just nitpicking. I get your point. Maybe my ideas might help. I don't think there's any need to post anything back for changing alignment.

That's not at all what I said! Changing the image implies changing the src attribute (and the widthand height attributes too). Of course that must be supported. RE: "a unique ID and storage context": in Drupal, each uploaded file gets a unique ID indeed (but if the same file is used multiple times, it's still referred to by a single ID). By "storage context" I think you actually mean "usage context", i.e. in which piece of HTML or which "field" of a piece of content this file (with a unique ID) is used, and this, too, is supported by Drupal: a DB file_usage table is maintained in which (file ID, content ID, count) tuples are stored. However, this table is updated when saving content, not when editing content.

(Note that in the case of Drupal, "fields" that are in fact just a piece of text, in this case HTML — it could also be Markdown or whatnot — are never parsed further out into more structured data. Drupal's "fields" are the atomic building blocks. So a "body" field that contains the body of an 2,000 word article does not get split into more things, nor do things get extracted from it. Instead, what would happen, is that you have e.g. "image" fields that are hidden from the end user, but are referenced from the body. Drupal builds and assembles HTML from a data structure, it doesn't disassemble HTML into a data structure. I think most CMSes work like this, but I of course may be wrong :).
I think this may part of why you think it's necessary to always talk to the server; if structured data is derived from the HTML you write with CKEditor, then I understand your reasoning, but in the case of Drupal, this would never happen; it'd be the other way around.)

To each his own. I could use to have "global" widgets where their data is stored globally. I can't argue with you, but don't you also have that in Drupal? Objects with global ids or names, such as galleries, menus etc... Will you not edit them in wysiwyg? Or you'll allow to drop them in and then go in the backend to edit them? Or just never allow them in an editor?

I do need to deconstruct the widget on post-back, if I am to achieve this. Please also see my notes about cross-referencing features at the bottom of this post, that is a great area of interest to me.

I also find this contradictory, you say "this table is updated when saving content". Doesn't that count as disassembling the HTML produced by the editor? Somehow, I think we are talking about the same workflow here. I find always connecting back to the server natural when you change something in the "properties" box, that counts as structured data. I also find opening the properties box, locating the input, changing its value and clicking Ok takes me around 2-3 seconds. I don't see how the 0.5s that should be a normal roundtrip time to refresh the layout would make such a big difference.

Copy/pasting Widgets we did not even consider (yet). I'm not sure how you'd pull that off. After all, the Widget is rendered in a certain way, and I don't think it's possible to intercept the "copy" event so that you would be able to pass it structured data instead of the exact HTML? Nested Widgets sound pretty insane and I think we can only discuss that with a reasonable confidence level once the non-nested version is up & running.

Na, ideas can't be insane, only their owners. People around already know I'm a bit insane. However, the amount of insanity involved is not for me to judge, I best leave it for the people that made the IE6 support in FCKEditor to decide... see, there's also such thing as good insanity. I think it can be done and will (eventually) be done. If it can be implemented so that nested widgets are an incremental step from non-nested widgets, it would be great. If not, so be it. No harm in exploring the possibilities.

Oh, and I wouldn't pull anything off, I'm just politely asking people to pull it off for me. I have the "what" and "why", but very rarely the "how". I just think, if the "what" and "why" are good enough, maybe a "how" can be found. Whenever I get a bit of "how" myself, I just relax and bump my chest with excessive pride. I try to use "I think, maybe, could it be...". If I ever did otherwise I appologize, I'm none the wiser.

3) "If you instead assume a single function/object/whatever being responsible for the rendering, then all that complexity goes away". With me it's a bit of an understatement. If you allow for nested widgets. While you have one widget per DOM node, the deserialization scheme will be nothing simple. Good planning can't hurt if you are ever to support such feature.

This paragraph could also be clarified by cleaning up the language. It's very hard to understand what you mean exactly :(

What I meant to say is (using my own convoluted grammar) that the problem is only simple if widgets are completely disjoined. Because then, their transformation can be handled by iterative, global events, like you envisioned. But if you allow any kind of widget interaction (such as nesting or reciprocal update), the problem is bigger because the event model would be a lot bulkier. One widget will need to be aware of the widgets nested into him (or that it is nested into?).

In an event-driven thinking, I take it that by "we could have more than one part of the code touching the final rendered version of widgets" Fred complained that each node might generate multiple events, with possibly multiple, possibly inter-related, handlers. You fixed it by saying each node will trigger one event with one handler. I un-fixed it by pointing out there may be situations (such as nesting) where a source transformation event might need to be bubbled (or sifted? who knows, W3C have just recently made their mind that they need both, and they weren't even faced with such a complicated challenge).

I don't know how this should be done at this point (I'm not that smart), but I learned that even outlining the problems might help, lest someone find a way to overcome them. I'm like a man pulling a cart without wheels. I'll just keep complaining until someone invents the wheel. Then I'll say it was my idea.

I do think that some form of event-driven, context-aware, bidirectional content transformation could address fix copy-paste problems, nesting widgets, and the content transformations you need, and (from what I read from #9829), paste from word. Except #9829 is one-way only. I think data extraction is by no way as problematic as insertion. So it'll have to work :)

I agree that thinking the Widget feature through is very important. But can you really think of a case where nested Widgets are a vitally important feature? Can you think of a way to implement nested Widgets in a way that will result in reliable, sane code? I can't see that just yet.

IMHO it makes more sense to implement an easy-to-use, easy-to-implement Widgets API at this stage, and if the need arises, implement a more advanced version in the future. Simplicity is important if we want this to take off and if we want people to build many Widgets and experiment with it.

My examples have long been in native text editors, so I don't think I will come up with anything revolutionary here:

1) "References list" widget with "Reference item" nested widgets (yes, with metadata like author, ISBN etc. stored and indexed globally, don't think it's insane)

• The scientific authoring community may take a keen interest in this one, with paper books becoming so obsolete
• Maybe some governmental agencies that spend on DMS systems that automatically and badly cross-reference MS documents may be interested in an alternative, at least for their public disclosures.
• How about those online law libraries? This is what they sell, cross-referencing. They might find such features useful, if they can be used in a wysiwyg manner.

2) "Resource" widget inside "Picture with caption" widget; pardon my XML:

<foo:pictureWithWidget pictureId="134">
  <foo:caption>
    I was married to <foo:resource id="3424" type="person">John</foo:resource> for only 2 weeks and on our <foo:resource id="4343" type="gallery">honeymoon</foo:resource> when we took this picture on <foo:resource id="23432" type="geographic">Mount Rushmore</foo:resource>
  </foo:caption>
</foo:pictureWithWidget>

Do I see reliable, sane code server-side? Definitely yes. Sane code editor-side? Yes, that's kind of the point of this discussion. Reliable code editor-side? No. But I hear at some point even setting a <strong> tag was not that reliable. I couldn't appreciate how this should be done or what is the level of insanity required on the editor side. I haven't done any development on CK. It's a black box for me, with a gray, nice, new front-end. But what I love most about these features is that they're bound to happen. You can only be too early, or too late, but you can't be wrong to plan for it. Hope google doesnt get there first :)

[Wim Leers]

Replying to dinu:

I suspect we're misunderstanding each other :)

Replying to Wim Leers:

1) For me, Fred is making perfect sense the best way to go is to have widgets re-rendered server-side. What JS can mirror of Drupal (or any other CMS) without any AJAX calls is semantically small-talk.

Please improve this sentence because it is hard to fully understand.

Well, the whole point of having the function of your CMS behind (with templates, filters, what-not), is to generate semantically difficult content, using many resources stored and cross-referenced on the server. I don't see how one could mirror that in javascript. I tend to believe this was also what Fred meant by "Due to the extreme flexibility of Drupal (modules, filters, hooks, etc.), we could have more than one part of the code touching the final rendered version of widgets". It's all a matter of how you percieve these widgets. I see them taking every bit of your CMS into the editor, giving you the possibility to include your site into itself, create a time paradox and be killed by your twin self from the future. Or just have the features MS-Word has and Google Docs are trying to build. I don't think we are barking at the wrong tree here, this discussion is (for me) what sets CKEditor apart from others. I take it from your view that you see the widgets as something-like-but-not-quite configurable HTML templates. I say "not quite" because you do want semantic CMS data produced, not just plain HTML. Functionality and usability are often seen throwing punches at each other, I guess.

I'm sorry, but your writing is still very hard to understand. You jump from aspect A to B to C very quickly, using metaphors along the way. I like succinctness and metaphors, but in this case it's hard to understand. Of course, that could just be a problem on my end :)

What I agree with in the above is this: "It's all a matter of how you percieve these widgets.". That is very much true! You go on to say this: " I take it from your view that you see the widgets as something-like-but-not-quite configurable HTML templates." — correct. Overall, it seems you just want to enable very advanced stuff. With that I of course also agree. We haven't had the time yet to look into very advanced stuff, so the most "advanced" thing we did so far was essentially this:

• Drupal has this concept of ​tokens.
• It is possible to embed a token in a piece of text, e.g. Hello, [user:name], and welcome to [site:name] on this lovely [date:day]!. When this gets rendered for the end user, these placeholders will be replaced with the corresponding contextual values.
• We made each of those into Aloha Blocks (cfr. CKEditor Widgets), made them clickable, and upon clicking them, they would become <select>s (retrieved from the server) that list all available tokens, where you would then be able to select another token instead.
• It is this server interaction that is one example of why you think it's essential to have the ability to let the server handle CKEditor Widgets. I don't disagree with that! All I am saying is that it should also be possible to change properties of Widgets without going back to the server, and see the (re-rendered) result immediately.

Another example: if you e.g. want to give the author the ability to insert an ad in a certain location without inserting the specific ad HTML code, because one year from now, that HTML will likely look different even from the same provider, but it's also rather likely to hop from one ad service to the next.

Essentially: have both an internal representation (i.e. the canonical HTML representation) and an external representation (i.e. the user-visible HTML representation). The mapping/transforming from the internal representation to the external representation would need to be implemented both on the server-side (for full fidelity, since this is what the end user will see) and on the client-side (for best effort fidelity, since it is e.g. often disallowed to show ads to site owners, or because it is too complex, or whatnot).

What is clear though, is that you're in favor of "let the server render a widget after every change of a property". I cannot stress enough that this is an absolute nightmare for performance. Please see comment:19.

Come on, it's not a nightmare, it's a dream. Honestly. It's a feature that noone has implemented yet. So it's an absolute v1, right? I don't see how something that is an absolute first could be a nightmare for anyone. Except increased government taxes. And a huge meteor.

Its' fine if some Widgets want to use that, but not every Widget should have to do that. Or, even more granularly: not ever Widget property change should have to do that.

("That" being going back to the server to re-render the Widget due to a property change.)

The other actions are: change the caption text and change the alignment.

If you change the alignment property through the properties box, that can also set a data- attribute on the widget node when clicking ok... so no request to the server is necessary.

Eh … huh? That completely contradicts with what you've been saying above!? This is what I've been arguing for all along!

However, I don't think it's a good idea to set an attribute directly. How could the code know where to set if the Widget can be rendered into any DOM structure imaginable?

(That's why in the "captioned image" Aloha Block, there's a default render callback implementation, but you can swap it for another implementation. The render callback receives something like

{
  original-image: '<img src="rocket.jpg" width="100" height="300" />',
  caption: 'Saturn V',
  align: 'left',
  width: '100'
}

which can then render it into whatever structure. Upon saving, the rendered result is replaced with

<img src="rocket.jpg" width="100" height="300" data-caption="Saturn V" data-align="left" />

)

Or what you need to achieve is transform a <span style="text-align:right"> into a data- property?

No :)

Imagine changing the "alignment" property and having to wait >2 seconds for this to take place… that would be horrible UX.

If the roundtrip is 2s, the user will have a lot more to complain on than the latest feature of the editor, I think...? Just nitpicking. I get your point.

Well, actually, that's not really true. This is a valid use case.

Imagine you're working in a very remote location, where there's no cable/DSL internet connections. Only EDGE, 3G or satellite. Latency will be very high, but you still need to get work done, for example because you're part of a team helping in a disaster area (say, Hurricane Sandy), or for example because you live somewhere in Africa where there's fine EDGE/3G coverage (and you pair your smartphone with your laptop for internet access) and you need to write a report on the company/government intranet. Then it is acceptable to wait, say, 1 minute for the page to load if you can work fluently afterwards. But if for certain things there are callbacks to the server that impede you from working fluently, then there is a problem. A more common scenario for you and me would be: working on the road where there is no Wi-Fi, so you connect with 3G from your smartphone. That's also often got a 1 second latency, and the same reasoning applies there.

I understand that you're saying you're just nitpicking, but really, this can sometimes make a huge difference! A few weeks ago there was Hurricane Sandy that hit New York City really bad. But the MTA (Metropolitan Transportation Authority) site (a Drupal site) was being updated by MTA employees with hyperlocal information from their smartphones on 3G. It was slow, but it worked, and it was absolutely essential for millions of citizens!

To each his own. I could use to have "global" widgets where their data is stored globally. I can't argue with you, but don't you also have that in Drupal? Objects with global ids or names, such as galleries, menus etc... Will you not edit them in wysiwyg? Or you'll allow to drop them in and then go in the backend to edit them? Or just never allow them in an editor?

Yes, we do have "global things". In fact, that is very much the Drupal way: structured data, that can be combined, correlated, etc. in any way. In text fields (where we would use WYSIWYG editors), this structured data is only "referenced" (synonyms: "embedded", "inserted").

So, no, we never allow the user to edit them in a WYSIWYG editor and then have the canonical data updated. Instead, one must update the canonical data, because it's an independent "thing". However, it's certainly possible that we'll want to do something like that in the future.

In fact, the end goal as I (and several others) currently see it for Drupal is essentially this: a unified syntax for referencing other content within and outside the current Drupal site. For example, something like <macro provider="local" type="node" id="345" language="pl" />, <macro provider="local" type="image" id="96" data-caption="Saturn V" data-align="left" /> and <macro provider="youtube" type="video" id="ef65fd98765" />. Something like that will not happen until Drupal 9 (several years from now) though, unless in "Drupal contrib" (contributed modules). It would then become feasible to have a single "Drupal Macro CKEditor Widget" plugin for CKEditor which would make it trivial for other people to allow more things to be embedded/references in textual fields, and still provide the ability to get live previews and the ability to render it in whatever way each site wants.

I do need to deconstruct the widget on post-back, if I am to achieve this. Please also see my notes about cross-referencing features at the bottom of this post, that is a great area of interest to me.

I also find this contradictory, you say "this table is updated when saving content". Doesn't that count as disassembling the HTML produced by the editor?

Yes, you're absolutely right there! :)

This disassembling is messy, and that is precisely why Drupal doesn't do this kind of thing out-of-the-box yet. The web's way of linking (i.e. referencing) is through hyperlinks (<a href="something">link to something</a>). That works fine, but it requires a lot of parsing. Hence our inclination towards a common syntax for referencing anything, which would make tracking cross-references far more efficient.

Somehow, I think we are talking about the same workflow here. I find always connecting back to the server natural when you change something in the "properties" box, that counts as structured data. I also find opening the properties box, locating the input, changing its value and clicking Ok takes me around 2-3 seconds. I don't see how the 0.5s that should be a normal roundtrip time to refresh the layout would make such a big difference.

I also think we're more or less on the same page, but I think we're doing or looking at some things differently. E.g. anything in a textual field we do not consider structured data in the sense that indeed you can parse it, but that's inefficient, so Drupal tries to store as much as possible in "clean data structures" such as DB tables.

An example. You can configure Drupal to have a "blog post" content type. This content type would have a "title" field (plain text, i.e. all HTML tags get stripped), a "body" field (rich text, i.e. with filter processing) and a "file" field (for images, PDFs, etc.). Imagine that a blog post wants to use a screenshot used in a previous blog post. Then you would make the "file" field hold the file ID of a previously uploaded file (when saved, this would then be stored in the file_usage table I mentioned before). This is the structured data part and the referencing. However, it's not yet in your body field, and that's where you want your screenshot to appear. So now you can click the "Insert" button next to the image file that you referenced in the "file" field, and it will append the basic HTML markup (<img src="screenshot.png" alt="a" title="b" width="200" height="200" />) to the end of the "body" field. As you can see, there is no strong connection between references inside rich text fields and things in the DB. We have to jump through hoops to get those strong connections.

Essentially, we consider everything that is "user-defined HTML" (and not HTML generated by Drupal) to be unstructured. Hence we would not consider any property in a CKEditor Widget Properties Panel to be structured data, and hence a round trip to the server makes zero sense for us.

Na, ideas can't be insane, only their owners.

:)

I think it can be done and will (eventually) be done.

Agreed. But it's not the minimally viable solution. Fred is arguing for KISS here. I strongly agree with that.

the problem is only simple if widgets are completely disjoined. Because then, their transformation can be handled by iterative, global events, like you envisioned. But if you allow any kind of widget interaction (such as nesting or reciprocal update), the problem is bigger because the event model would be a lot bulkier. One widget will need to be aware of the widgets nested into him (or that it is nested into?).

Agreed.

But what do you mean by "reciprocal update"?

but I learned that even outlining the problems might help, lest someone find a way to overcome them. I'm like a man pulling a cart without wheels. I'll just keep complaining until someone invents the wheel. Then I'll say it was my idea.

Haha :D Did you start drinking somewhere along the way of writing this? Wittiness levels are increasing towards the end, it seems! :D

I do think that some form of event-driven, context-aware, bidirectional content transformation could address fix copy-paste problems, nesting widgets, and the content transformations you need, and (from what I read from #9829), paste from word. Except #9829 is one-way only. I think data extraction is by no way as problematic as insertion. So it'll have to work :)

The "bidirectional content transformation" part I can relate to. I'd be fine with exploring whether it's worthwhile to have not only a render callback but also a parse callback. Because I can definitely relate to the fact that Drupal may be the exception in wanting the widgets to have both a canonical/internal representation and a rendered/themed/external representation. If not every system wants such an canonical/internal representation, then it's likely necessary by design to also have a parse callback for each Widget.

[Dinu]

So, it seems we want the same things, just not at the same time. Works for me.

Point is, yea, we're not gonna get time travel... this year. But knowing where you're going certainly reduces overhead. I don't think there's a problem, I remember discussions about nested contentEditable (eg. widgets) strarted about 2 years ago. It was an insane idea back then. Now it's knocking on our door. Where is it with tinymce? Still in the crazy bin, I guess.

If you change the alignment property through the properties box, that can also set a data- attribute on the widget node when clicking ok... so no request to the server is necessary.

Eh … huh? That completely contradicts with what you've been saying above!? This is what I've been arguing for all along!

However, I don't think it's a good idea to set an attribute directly. How could the code know where to set if the Widget can be rendered into any DOM structure imaginable?

How could CK know that the image lies in a fixed-layout content? Probably, by means of contenteditable and some data-widget property. The point is, CK will know it's a widget, if it's to open a property box, and a root node for it. If CK knows, so will you. So you can set properties on it.

[Frederico Caldeira Knabben]

Ok, I think we already burned too much of the energy of Wim's and Dinu's brains here :) We may be going a bit off-topic, but still all the talk brought ideas and concepts that need to be considered, either now or in the future.

Our goal though is having something available still for CKEditor 4.1. You guys saw how huge our TODO list is, so we're definitely KISSing here:

1. Updating the rendering on properties change will be as simple as a callback. It'll be up to the implementor to decide whether to do it in the client or server side.
2. No focus on nested widgets.
3. No focus on input parsing for now. Still doable currently though, by data processor filtering.

We want to bring this feature to the market soon and then see the outcome (v1 as Dinu said). We'll then have plugins being created for simpler widgets and other needs may be brought to light.

Meanwhile, we'll be stabilizing the feature. Later on, we'll talk about the results and discuss more, to bring enhancements to it (v2).

[Dinu]

Can't wait for 4.1 :) I won't even write to Santa this year, he's already delivered :)

[Wim Leers]

fredck++ dinu++

:)

[Piotrek Koszuliński]

Related ticket: #9324. Magicline shouldn't be displayed inside widgets.

[Olek Nowodziński]

Latest changes bring some new features to the widget system. At the moment:

• widgets can be inline
• widgets can be used inside of framed editor instances
• editing system is completely based on dialogs

​See t/9764 branch on Github.

Some of the features can be unstable due to selection system issues.

[Matti Järvinen]

Good work looks very promising.

This feature is close what was discussed in ticket:3306#comment:6

Akismet doesn't allow to follow tickets with cc without any comment, sorry for the mails.

[Olek Nowodziński]

Latest changes bring accessibility enhancements:

• Widgets can be selected with keyboard arrows (with blur).
• Fixed selection on widget cut. At the moment selection is moved to the closest editable space, just like regular cut.
• Fixed selection on widget delete (backspace); same selection concerns solved.
• Selected widgets work with keyboard CTRL+C and CTRL+X.
• Widget selection can be restored i.e. when loading an undo image.

Tested with latest Chrome. Some features are still unavailable in FF.

​See t/9764 branch on Github.

[Wim Leers]

All my testing feedback was done in plugin/widget/samples/widget.html, commit 4ec05d9686182e6aa43246fdd085dcd6beee1b5a.

• Inline widgets: great!
• Framed editors support: great!
• Keyboard navigation across widgets & manipulation of widgets works great! :)
• Undo/redo support: works well, but there seem to be a few weird behaviors. For example, if you double click the "Captioned Image" widget in the "Captioned Image Widget Sample", set the alignment to left (instead of right), click ok, then click in the text. Now press CTRL+Z. The alignment is not undone, but instead the "Captioned Image" widget is selected. Press CTRL-Z again, and now it is undone. It seems "widget selection" is considered a step in the undo/redo process?
• Copy/pasting of the "Time" widget works flawlessly (even when using the keyboard only), pasting the "Captioned Image" widget causes JS errors, copying the entire "Quote" widget is very tricky (it seems I was somehow frequently missing the blockquote tag when copying) and I couldn't manage to select the "Video" widget at all.
  Overall it seems like it's very hard to select a block-level widget correctly for cutting/copying. I also think this is not the most critical thing to be working on right now — that can be improved at a later stage.
• Bug: If you go to the "YouTube Video Widget Sample" and click just after "UTC.", then you only have to press the right arrow once to activate the "Video" widget. Now press the "enter" key to open the widget's dialog. If you now press the "tab" key to jump to the "alignment" setting, you will in fact be navigating the page, not the dialog. For the "Time" widget, this problem does not exist.

Overall, it's already working very well! However, the single most important thing for this to become usable by CMSes in general and Drupal in specific is to have the functionality outlined in comment:12, bullet 2 and clarified in comment:18. If we'd have to save HTML like <div data-widget-wrapper="" class="cke_widget_wrapper" data-widget-id="" …> or even if the generated HTML is not overridable, then we would not be able to use CKEditor widgets at all. (Again, it's fine to have any kind of HTML while editing, but the HTML must be clean when it goes back to the back-end.) Unless I'm missing something, I don't think progress has been made in this area yet? I'd prefer to see work on this sooner rather than later because it might require several iterations to get it right.

If you want to address basic functionality and a11y concerns first (which it seems you're doing, considering comment:31 and comment:34), that's fine, but I'd like to have some idea of when and how you want to tackle these fundamental requirements.

[Olek Nowodziński]

Replying to Wim Leers:

All my testing feedback was done in plugin/widget/samples/widget.html, commit 4ec05d9686182e6aa43246fdd085dcd6beee1b5a.

• Inline widgets: great!
• Framed editors support: great!
• Keyboard navigation across widgets & manipulation of widgets works great! :)

This is the kind of feedback we love ;)

• Undo/redo support: works well, but there seem to be a few weird behaviors. For example, if you double click the "Captioned Image" widget in the "Captioned Image Widget Sample", set the alignment to left (instead of right), click ok, then click in the text. Now press CTRL+Z. The alignment is not undone, but instead the "Captioned Image" widget is selected. Press CTRL-Z again, and now it is undone. It seems "widget selection" is considered a step in the undo/redo process?

Confirmed. This is the problem we're facing right now and we know about it. Since we must cheat browser's selection system when selecting widgets, an empty undo snapshot is being saved (because we manipulate the selection). This issue will be fixed once selection is working.

• Copy/pasting of the "Time" widget works flawlessly (even when using the keyboard only), pasting the "Captioned Image" widget causes JS errors, copying the entire "Quote" widget is very tricky (it seems I was somehow frequently missing the blockquote tag when copying) and I couldn't manage to select the "Video" widget at all.
  Overall it seems like it's very hard to select a block-level widget correctly for cutting/copying. I also think this is not the most critical thing to be working on right now — that can be improved at a later stage.

Regarding caption widget and block quote widget: WFM. Tried both in latest Chrome and FF, copying and pasting frantically everything, everywhere. Please, specify the test case since I'm unable to reproduce it.

Regarding video widget: Also WFM in Chrome. FF doesn't copy iframe tags to the clipboard. This is why there are errors :b We need to cheat it somehow (e.g. serialize and deserialize iframes, change tag names etc.). There was a similar issue when FF refused to copy elements with display:none and custom attributes like foo="bar".

• Bug: If you go to the "YouTube Video Widget Sample" and click just after "UTC.", then you only have to press the right arrow once to activate the "Video" widget. Now press the "enter" key to open the widget's dialog. If you now press the "tab" key to jump to the "alignment" setting, you will in fact be navigating the page, not the dialog. For the "Time" widget, this problem does not exist.

AFAIR it might be more than widget issue. I remember that we faced similar thing a couple months ago with some other dialogs and other elements. Fortunately this is a minor issue.

Overall, it's already working very well! However, the single most important thing for this to become usable by CMSes in general and Drupal in specific is to have the functionality outlined in comment:12, bullet 2 and clarified in comment:18. If we'd have to save HTML like <div data-widget-wrapper="" class="cke_widget_wrapper" data-widget-id="" …> or even if the generated HTML is not overridable, then we would not be able to use CKEditor widgets at all. (Again, it's fine to have any kind of HTML while editing, but the HTML must be clean when it goes back to the back-end.) Unless I'm missing something, I don't think progress has been made in this area yet? I'd prefer to see work on this sooner rather than later because it might require several iterations to get it right.

Yep. I think you're missing a lot ;) But it's not your fault since we totally forgot to mention about it. Open widget sample and type in your console:

CKEDITOR.instances.editor1.getData()

See that there are neither widget wrappers nor extra attributes here:

...
<figure class="caption" data-widget="caption" style="float: right"><img alt="" data-widget-property="image" src="../../../samples/assets/sample.jpg" /><figcaption data-widget-property="caption"><a href="http://en.wikipedia.org/wiki/Saturn_V">Saturn V</a> carrying Apollo 11</figcaption></figure>
...

Also when you call a particular widget:

CKEDITOR.instances.editor1.widgets.instances[ 1 ].getHtml()
>>> "<span class="time" data-widget="time" timestamp=1357220873242 utc=false seconds=false>14:47</span>"

A clean widget is returned. This is because widgets are based on models. When you call getHtml on the widget, it uses it's own template (the same that is used for inserting brand-new widgets), to output previously gathered data. The same thing is done by the dataProcessor when executing editor.getData().

It's been implemented for a while but we missed the moment to let you know. Sorry for that ;)

If you want to address basic functionality and a11y concerns first (which it seems you're doing, considering comment:31 and comment:34), that's fine, but I'd like to have some idea of when and how you want to tackle these fundamental requirements.

What I did here is a very, very basic integration of widgets with an editor. Without it, editors used to lose focus and selection when navigating with the keyboard around widgets. This is much more about "not destroying the editor" than "providing a flexible accessibility". Our intention is to keep things simple in the very first version of widget plugin so we won't go beyond this kind of "features".

[Wim Leers]

Replying to a.nowodzinski:

Regarding caption widget and block quote widget: WFM. Tried both in latest Chrome and FF, copying and pasting frantically everything, everywhere. Please, specify the test case since I'm unable to reproduce it.

First: I should have mentioned that I'm doing all my testing in Chrome 23.

It's very well possible that I'm failing to select things correctly, which is then causing copy/paste problems.

I think the best way to get a reliable/usable list of steps to reproduce is by recording what I'm doing in a screencast in which also all of my keyboard and mouse actions are recorded?

CKEDITOR.instances.editor1.getData() }}}

Woot! That is awesome! Concern addressed!

If you want to address basic functionality and a11y concerns first (which it seems you're doing, considering comment:31 and comment:34), that's fine, but I'd like to have some idea of when and how you want to tackle these fundamental requirements.

What I did here is a very, very basic integration of widgets with an editor. Without it, editors used to lose focus and selection when navigating with the keyboard around widgets. This is much more about "not destroying the editor" than "providing a flexible accessibility". Our intention is to keep things simple in the very first version of widget plugin so we won't go beyond this kind of "features".

Sorry, upon rereading my comment, it could be interpreted as negative or downplaying what you've already done. That was definitely not my intention.

I was only worried that the Widgets feature would be written in such a way that we wouldn't be able to use it, but that's clearly not the case. No more worry now :)

[Frederico Caldeira Knabben]

A sub-ticket has been opened for the fake-selection feature, which is required by this ticket: #9982.

[Olek Nowodziński]

Squashed and force-rebased ​t/9764.

[Piotrek Koszuliński]

Rebased t/9764 branches on t/9982.

[Piotrek Koszuliński]

Time for update :)

For last 3 weeks with break for 4.1.1 release I was working on refactoring widgets implementation which started to grow in wrong directions :D. Few decisions we made at the beginning, when making prototype were wrong and in fact there was no other option than heavy refactoring (sometimes called writing from scratch).

I focused on data handling and widgets initialization/destroying. Nearly all code handling UI was disabled for now and we'll work on it after finishing data API.

Old widgets definitions are now outdated and I created new sample in plugins/widgetimage/samples/widgetimage.html.

3 editors visible in this sample have differently configured image widget. First uses the standard configuration, second one an upcasting method called captionedImage (only image width data-caption attribute will be recognised as widgets) and in the third editor image upcasting method is used which turns all images into widgets.

Reverse way is also more or less ready - in all 3 editors widgets will be downcasted back to their original forms when getting data.

Most of the design work has been already done, however this is still early implementation, so few things are broken. E.g.:

• On undo/redo widget loses its float style - I need to keep it inside widget in some stringified state object so it isn't lost. Most likely I'll use JSON (available since IE8, but widgets won't supported on IE7) to store widget's data.
• When switching between modes in 2nd and 3rd editors additional empty paragraphs are created. I need to fix DOM structure which becomes incorrect when upcasting widgets (initially parser wraps image in paragraph, but figure cannot be placed in paragraph).

During the following days I'll focus on missing fragments of this puzzle like these 2 points. Then I should be able to finally focus on things like UI and nested editables.

[Wim Leers]

Cool! :)

I looked at the commits and the example at plugins/widgetimage/samples/widgetimage.html. I don't think this is the right time to provide more feedback?

[Piotrek Koszuliński]

Branch is definitely under a heavy development so many things can quickly change and the implementation may not be clear. I think that the most interesting things you'll find in widgetimage/plugin.js and its sample. But I'm changing the plugin file a lot right now so it indeed isn't a right time to check it :D. However, it should get more stable after today/tomorrow - I'll notify here about this.

BTW. This:

When switching between modes in 2nd and 3rd editors additional empty paragraphs are created. I need to fix DOM structure which becomes incorrect when upcasting widgets (initially parser wraps image in paragraph, but figure cannot be placed in paragraph).

Is already fixed.

[Piotrek Koszuliński]

On undo/redo widget loses its float style - I need to keep it inside widget in some stringified state object so it isn't lost. Most likely I'll use JSON (available since IE8, but widgets won't supported on IE7) to store widget's data.

OK. Now also this point is fixed. I implemented data storage for widgets with serialization to element attribute, so data can be restored after undo/redo/paste/etc.

I made more changes to widgets API, but now that part of it which is currently used in image widget should be pretty stable. Any feedback is greatly appreciated :).

[Wim Leers]

I talked to Reinmar about which feedback he needs: just feedback on upcasting/downcasting (i.e. the second example). I tested that in detail, and AFAICT it works great. Combined with the other two examples, I believe every use case is supported.

The very small amount of code in plugins/widgetimage/plugin.js also suggests that for this (crucial) part of a CKE Widget's functionality, the code does not have to be too complex, which is great!

[Piotrek Koszuliński]

We decided to give widgets one more month to reach fully mature level of this new (and huge) feature.

[Piotrek Koszuliński]

I updated ticket description with current status of things. If I forgot about something I'll add it later.

[Piotrek Koszuliński]

1. Updated description.
2. Force-pushed t/9764 branches on latest major.
3. Two bugs fixed.

Some stats:

• Dev branch contains more than 200 commits and this is after squashing 100 of them.
• 6000LOC of tests have been written.
• It takes more than 20s on FF to run just these tests ;<.

[xmo]

Some feedback: the inline parameter is really annoying, especially when a widget can work both inline and block without any trouble. Since CKEDITOR.dtd already provides $inline and $block categories, could it be null by default (or removed altogether) indicating this will be evaluated at runtime and the right wrapper will be created on the fly depending on the widget's root?

Also the editor-level "change" event does not seem to trigger anymore on t/9764, whether inside or outside a widget. Not sure if the problem is in my usage of it.

[Piotrek Koszuliński]

Replying to xmo:

Some feedback: the inline parameter is really annoying, especially when a widget can work both inline and block without any trouble. Since CKEDITOR.dtd already provides $inline and $block categories, could it be null by default (or removed altogether) indicating this will be evaluated at runtime and the right wrapper will be created on the fly depending on the widget's root?

Yes, we also found it problematic. Your idea makes sense and may be a good solution. We'll think on it.

Also the editor-level "change" event does not seem to trigger anymore on t/9764, whether inside or outside a widget. Not sure if the problem is in my usage of it.

I was working on widgets integration with undo manager recently (the end of last week). I checked now and it seems that change is fired correctly when editing widgets, content inside nested editables and everything else. I think that you're testing older version.

Thanks for your feedback.

[Olek Nowodziński]

Replying to xmo:

Some feedback: the inline parameter is really annoying, especially when a widget can work both inline and block without any trouble. Since CKEDITOR.dtd already provides $inline and $block categories, could it be null by default (or removed altogether) indicating this will be evaluated at runtime and the right wrapper will be created on the fly depending on the widget's root?

I strongly support this idea!

[Piotrek Koszuliński]

The moment has come! Will it have R+?! :D

[Olek Nowodziński]

Replying to Reinmar:

The moment has come! Will it have R+?! :D

I bet 10 bucks it's gonna be R- :P

[Olek Nowodziński]

Replying to fredck:

Oh, come on! :P

[Piotrek Koszuliński]

I'm waiting for my 10 bucks. See you on major, guys!

[Piotrek Koszuliński]

Merged to major with ​git:22f3d33 on dev and d3f9e00 on tests.

Some stats:

• tests: 52 files changed, 10256 insertions(+), 337 deletions(-)
• dev: 38 files changed, 4846 insertions(+), 626 deletions(-)

[Matti Järvinen]

Great job!

[Wim Leers]

Wow! Great job! :)