Text direction is not preserved for Paste function

[Amir]

When using paste function thorugh paste dialog, the text direction is not preserved from dialog to editor. The expected behavior is to see the text after the paste the same way as it was shown in the paste dialog.

Notice in the attached image that text as displayed in paste dialog has different BTD from how it presented in the editor.

Note: Paste dialog is only avaialble in FF and Chrome browsers.

[Tomer Mahlin]

Steps to reproduce:

1. Invoke paste dialog
2. Type a text into the dialog (i.e. "hello world !")
3. Change direction of text inside the dialog using browser means to RTL (i.e. for IE use <Ctrl>-<Right-Shift>, for FF use <Ctrl><Shift>-X)
4. Observe how text is shown in the dialog
5. Press OK
6. Observe how paste text is shown in the editor

Observations: Text after being paste into editor assumes the same direction as paragraph part of which it becomes. Instead, it is expected that it will preserve the same direction as was assigned in the Paste dialog. This can be achieved by using Language:LTR / Language:RTL styles which CKEditor already supports. In addition to cleanly separate the injected segment from the surrounding text a markup (i.e. BDI) can be used.

[Jakub Ś]

1. When you do that dir attribute gets assigned to body which isn't included into paste. Content that gets pasted doesn't have any information that it is RTL.
2. I was trying to find more real-life test case as above is rather something you simply don't do. To do that I have assigned dir="rtl" to body tag in plain HTML5 page. Next I have copied contents into editor and plain editable page.

Both results are the same - compied content doesn't contain body tag, just what is inside it. And contents of body tag don't have any information of being set to RTL.

If pasted content e.g. main div had dir="rtl" and allowedContent was disabled or set to allow such attribute there wouldn't be a problem.

[Teresa Monahan]

@j.swiderski - would it not be possible to get the dir attribute from the body of the paste dialog and apply it to the contents if it is different than the expected dir? This information should all be available inside the commit function of the paste dialog.

[Jakub Ś]

In second case only div is passed by browser, there is no body so there is no dir attribute and there is no way to know if it should be there or not.
In first case, where RTL is set directly in paste area, this isn't typical use case but since this is happening in editor it could perhaps be done.

Looking at the code of clipboard/dialog/paste.js it looks like some attribute rewriting could be done in commit method. I will however need to ask @fredck about this. I personally have doubts (despite it sounds reasonable) if editor should assign attributes to elements that didn't have them.

[Frederico Caldeira Knabben]

Replying to tomerm:

Steps to reproduce:

1. Invoke paste dialog
2. Type a text into the dialog (i.e. "hello world !")
3. Change direction of text inside the dialog using browser means to RTL (i.e. for IE use <Ctrl>-<Right-Shift>, for FF use <Ctrl><Shift>-X)
4. Observe how text is shown in the dialog

Unfortunately this is a hack to force a problem, but not a clean reprodution step. Point 3 may make the browser behave in a different way than real RTL text being pasted.

So, to make this a good ticket, we need to have steps that include copying real RTL contents. These steps could either point us to a website or having a document to be downloaded in this ticket.

[Tomer Mahlin]

Not sure if usage of standard browser / OS means is considered a hack :-))).

In some cases when RTL content is pasted into paste dialog its direction is preserved (try to do so copying text from CKEditor itself). In those cases we don't have any issues - after pressing OK button text is properly embedded into edited document and with proper direction.

However, in other cases this does not happen (for different reasons). In other words RTL content is pasted into paste dialog as LTR content. The only way to affect its direction is to use browser tools - key shortcuts mentioned above.

The only work around at the moment for the problem is to past RTL content into document as LTR one, explicitly select it and then change its direction to RTL already inside CKEditor. However this is very awkward and by all means not user friendly approach.

I see 2 possible approaches for problem resolution:

1. Trace browser keyboard shortcut and take it into account when text from paste dialog is embedded into document content.
2. Provide additional means affecting text direction in the paste dialog itself - for example 2 buttons similar to those used in the CKEditor itself for the same purpose

[Jakub Ś]

@tomerm you mention "different reasons" but there is simply one reason. If pasted content has not direction information than it is pasted as LTR.

For example if Paragraph has {<p dir="rtl">keep going, CKEditor</p> it will be RTL but if you paste such {<p>keep going, CKEditor</p> which is wrapped in RTL body then as i have already explained, such body is excluded from paste and pasted content has no information that it is RTL. Thus you get such result.

I agree with @fredck that this is kind of hack. If you need RTL content then please paste such content (pasted element(s) shout have this attribute) and don't change it in paste dialog as attribute gets assigned to body (browser does it) and not pasted element(s).

[Tomer Mahlin]

The reasons are different, the result is the same as you pointed out - if pasted content has not direction information than it is pasted with direction which is inherited from UI. Namely:

• if CKEDitor is not mirrored (this happens when it appears with UI in Latin languages) then direction is LTR,
• if CKEditor is mirrored (this happens when it appears with UI in Bidi languages such as Arabic / Hebrew) then the direction is RTL.

What are the reasons ? Those are associated with correlation between:

• the way web page was constructed (where specific text direction information was present). I presume that text is being copied from web page and later on pasted into CKEditor.
• the way text from HTML page is selected (before it is copied and then later on pasted into CKEDitor)
• the way direction information is passed with copied text into clipboard.

Obviously not only CKEditor but even browsers can't control all those 3 aspects. Thus it is not reasonable to assume that direction information will be magically present with the text being copied from a web page. However, as a simple user (who even doesn't realize what HTML is), I would like to see the same piece of text consistently on both HTML page (from which it is copied) and inside CKEditor (to which it is pasted).

I don't think it is a hack to copy some text from HTML page and paste it into CKEDitor. Would you agree that this is a valid scenario ? If so I would like you to help me understand what poor user should do to preserve proper display. You say that if "... I need RTL content then please paste such content - pasted element should have this attribute ...". As a simple user copying some piece of text from HTML page, how possibly can I assure that (pasted element to have direction attribute) ?

The only way to assure proper display after copy operation is complete, is to affect direction attribute of pasted element. Currently CKEditor does not allow to do that. What I am asking to is to allow end user to affect direction attribute of pasted element. This can be done in 2 possible ways as I explained in my previous comment.

[Tomer Mahlin]

To make my arguments above valid we need slightly change the test procedure. Specifically in step 2: Instead of:

Type a text into the dialog (i.e. "hello world !")

We should suggest to:

Copy some text from any web page and paste it into the dialog.

Relating to comment by: @j.swiderski "..I personally have doubts (despite it sounds reasonable) if editor should assign attributes to elements that didn't have them...". The element de facto (from display perspective) could have the attribute which was lost (again from display perspective) during copy / paste operation. We simply want to have a consistent display. Since environments in which piece of text being copied lives are different (original web page from which it is being copied and content of CKEditor into which it is being pasted) we may need some markup modifications to achieve consistent display.

Relating to comment by: @fredck "...we need to have steps that include copying real RTL contents. These steps could either point us to a website or having a document to be downloaded in this ticket. ..." Ok. Not a problem at all.

• Let us assume we are working with English text in CKEditor which is currently appear as translated to French and thus having not mirrored GUI. Let us assume we want to paste into it a real RTL text which we copy from a real RTL content.
• I suggest to use ​http://en.wikipedia.org/wiki/Ghandi entry in Hebrew which is available from: ​http://he.wikipedia.org/wiki/%D7%9E%D7%95%D7%94%D7%A0%D7%93%D7%A1_%D7%A7%D7%A8%D7%9E%D7%A6%27%D7%A0%D7%93_%D7%92%D7%A0%D7%93%D7%99. Look for second horizontal separator line. Select first 2 sentences from the paragraph which starts immediately after it.
• If I go directly and past this text into CKEditor inside LTR paragraph I get this text displayed with LTR direction. THis is expected since copied text lost (or in any case does not have) direction attribute and thus it assumes direction of paragraph into which it is embedded.
• When I do the same via paste dialog I get the same results - piece of text is displayed with LTR direction. I would expect paste dialog to provide more enhanced functionality in this contexts as compared to direct paste option.

One additional observation. There are buttons / options for Paste, Paste as Plain Text and Paste from Word on the toolbar. Probably we need to have additional option for Paste from HTML or "simple" Paste button already covers that ? If it does cover HTML already, I believe from Bidi (support for Arabic/ Hebrew) perspective it should make sense to enhance it's behavior and provide more advanced support as compared to direct paste option (not using any dialogs).

[Tomer Mahlin]

Apologies if it was not clear enough from the very beginning. The goal of user in described scenario is to assure that display of copied text and display of pasted text are the same (from text direction perspective). Thus by all means I don't insist on step 3 in the list of steps below.

1. Invoke paste dialog
2. Copy some text from any web page and paste it into the dialog.
3. Change direction of text inside the dialog using browser means to RTL (i.e. for IE use <Ctrl>-<Right-Shift>, for FF use <Ctrl><Shift>-X)
4. Observe how text is shown in the dialog
5. Press OK
6. Observe how paste text is shown in the editor

I listed it since from user perspective it was the only way to affect display of copied text (at least inside paste dialog). As I suggested later on, CKEditor might decide to provide alternative approach for affecting direction of copied text. For example adding two buttons to paste dialog which can explicitly set direction attribute on the pasted element. Whatever you believe is more appropriate.

We can also suggest to accept current functionality and suggest end user to select pasted text AGAIN, (this time in CKEDitor) and using standard CKEditor tools to affect it's direction. I believe user will find this approach problematic since:

1. Copying the same text twice is not very user friendly
2. Copying text with natural direction different from direction of paragraph into which it is embedded might not be trivial (it is not trivial to identify beginning and end of text to be copied).

[Jakub Ś]

I don't think it is a hack to copy some text from HTML page and paste it into CKEditor. Would you agree that this is a valid scenario?

But after that you press keyboard shortcut - this is what @fredck meant by sort of hack. You are using browser or OS feature to get specified behaviour. Btw. I don't think poor simple user knows such shortcut.

If so I would like you to help me understand what poor user should do to preserve proper display.

Certainly - you paste content and then select it and change it with BIDI buttons. That is what simple user who has no knowledge in HTML would do.

Please note that HTML pasting is contextual. It means that it will fit the styles of the context it is being pasted on.

The behaviour is valid (this is all how HTML works). If there is no information about the dir, pasted content will have same direction as the target content.
Paste dialog will not sniff language direction if there is no information (attribute).

The only place for improvement is copying this attribute from the dialog if it was changed with shortcut. This can be considered as feature but as I said before - users can copy/paste text and then change it with BIDI buttons.
I know users would like editor to read minds and do exactly what they want :) but sometimes some user effort is required.

To summarize: behaviour is valid but perhaps there might be some place for feature. @fredck could you perhaps comment on this?

[Tomer Mahlin]

Thanks a lot for the clarifications. Usage of keyboard shortcut (whether we call it a hack or not) is a well known to all simple users method to affect direction of text in browsers. Please observe that in simple (not rich text editor) plain text input fields on HTML page this is the only mean for the end user to affect direction of text. This is the reason they are aware of it.

I don't mind to call this ticket a feature if it helps achieving better user experience (this is a common goal in fixing defects and implementing features). What I am sure in is that:

1. Selecting piece of text after it was pasted into CKeditor is a non trivial task since in cases in which natural direction (not de-facto) of this piece of text is different from natural direction of surrounded text, it is not clear how to find the boundaries of pasted text. Thus, while in theory users can copy/paste text and then change it with BIDI buttons, in practice it is often a very challenging task.

2. In Bidirectional world or languages presence / coexistance / intermixture of English and Arabic (or English and Hebrew) sentences / text is a matter of everyday life. In light of that "Contextual" behavior of HTML in this context ("... If there is no information about the dir, pasted content will have same direction as the target content...") does much more harm than good.

In light of all that, I believe that it is very much desirable to introduce a feature which will allow copying text direction information/attribute from the dialog if it was changed with shortcut. I would appreciate if you consider it as a valid feature request :-))

Just FYI, I am leading a development team which can contribute code / functionality to CKEditor (we simply submit a patch and discuss it between us until we agree on the code which you eventually deliver). If it makes things easier we can provide a sample implementation for such a feature.

[Jakub Ś]

IMHO introducing feature that can be controlled with configuration option is good idea. I would like however to first hear @fredck opinion on this (I will forward this to him).

@tomerm you can always take CKEditor 4 code from ​https://github.com/ckeditor/ckeditor-dev, prepare your fix and make a "pull request" (here is the example list of pull requests ​https://github.com/ckeditor/ckeditor-dev/pulls?direction=desc&page=1&sort=created&state=open). In comment please specify link to bug for which this fix was prepared.

Such pull request can be accepted, rejected or used as start point to develop this feature.

[Tomer Mahlin]

Any updates from @fredck ? I would like to have a clearer picture before I commit the resources on my side and develop initial patch. Thanks.

[Moshe Wajnberg]

Hello

I just issued a new pull request for fixing this defect ​https://github.com/ckeditor/ckeditor-dev/pull/73

[Frederico Caldeira Knabben]

Replying to wajnberg:

Thanks for the ​PR. Unfortunately it doesn't handle the problem properly. I've added comments to it.

[Frederico Caldeira Knabben]

Let's come to a conclusion on the analysis of this issue.

The reason for the paste dialog to exist is just because browsers block clipboard access from JavaScript. Therefore, we needed a way for users to transport their clipboard data to the editor. The simplest solution was bringing a dialog with an input space, so the user could paste into it.

So, we're always talking about "paste". The intention of the dialog is not giving an input field for the user to "type" inside. In fact, the dialog clearly instructs the user to "paste" in that field using the keyboard and hit the ok button. That's what we want to support there and we'll hardly enlarge it with unrelated features. In your TC, other that typing, you're additionally editing the content directly in that field, in a non standard way.

For instance, we're taking in consideration replacing the dialog with a "field-less" message just asking the user to use the keyboard to paste, so the clipboard data will be grabbed in a different way. This means that, if we try to mess thing up today because of BiDi, the solution may get outdated at some stage, because it is incompatible with the reason why the dialog exists.

The only thing we can do is relying on the data that browsers sent to the us on paste. If the direction information is missing in it, it means that the end user will have to use the BiDi tools available in CKEditor to fix the content accordingly, by selecting the pasted text and applying the expected direction to it.

[Tomer Mahlin]

I think we already discussed the problem and reached an agreement about importance of the fix. Text direction usualy does not make into paste dialog during "paste" operation simply becasue in the source from which "copy" is made this information may be specified very differently. Suggested in your last comment solution is very awkward due to the way bidirectional text is displayed to the end user. Simply find the boundaries of pasted text to select it in CKEditor would be impossible mission. I understand your arguments on the dialogless way to paste data in the future. But I don't understand why implementing suggested solution will be in a way. If you remove the dialog you will remove Bidi feature altogether. There will be no need to continue to support it since dialog is gone. From Bidi perspective having message? way to paste content is very problemaitc from end user experience perspective.

[Frederico Caldeira Knabben]

Just note that, currently, pasting with CTRL+V brings the very same problem. So, the dialog is not the solution for it. In fact, in some browsers the dialog may not even show up altogether (e.g. IE when the user allows js to access the clipboard).

We understand that a problem exists here. We just need to find the proper way to handle it, which may be post-pasting processing controlled by users. This is something not available right now.

[Teresa Monahan]

Replying to fredck:

Just note that, currently, pasting with CTRL+V brings the very same problem. So, the dialog is not the solution for it. In fact, in some browsers the dialog may not even show up altogether (e.g. IE when the user allows js to access the clipboard).

We understand that a problem exists here. We just need to find the proper way to handle it, which may be post-pasting processing controlled by users. This is something not available right now.

Hi Fred, Thanks for your comments. The point you raise about this also been an issue for CTRL+V is very valid. However that is a much more dificult scenario to address than the paste dialog. Even if we only had a solution for this through the paste dialog for now, then at least BIDI users would have some way to 'fix' their content after pasting.

Your review comments on the pull request which @wajnberg provided are very helpful and you are of course correct in saying that the dir attribute should be applied at the block element level, in the same way as the BIDI feature of the editor. If @tomerm's team revise the fix based on these comments, would you we willing to review it again and deliver agreed changes into the CKEditor code? We would be happy to make this configurable and even disabled by default if that works better for you. It really would be good to have the possibility of enabling some sort of a workaround for this for BIDI users where needed though.

Thanks, Teresa