Bring back FCKConfig.EnableXHTML and FCKConfig.EnableSourceXHTML

[Jeff Fohl]

Any chance of bringing back the FCKConfig.EnableXHTML and FCKConfig.EnableSourceXHTML options? The forced conversion to XHTML of source code causes no end of headaches. I am not clear on what the reasons for deprecating these are.

[Alfonso Martínez de Lizarrondo]

Could you please list the problems of the current system?

[Jeff Fohl]

Thank you for responding so quickly. Here is a list of some of the problems that we have. Just so you know, we are a company that develops a Content Management System for web publishers, and we have the commercial license for FCKEditor, which is currently embedded in our software. Our users have a great range in technical skill - from completely ignorant of HTML to advanced programmers.

Here are some of the problems that we have with the automatic conversion in to XHTML:

• Automatic conversion is unintuitive. This is probably the number one issue. Most users are not aware that this is happening. When they toggle from the Source view to theWYSIWYG view, and then back - their code has been modified, and they don't know why. Worse - they sometimes do not know that it has occurred and only see that the result is not what they had intended.

• The Cascade Effect. When the FCKEditor attempts to fix errors, it sometimes will re-write the entire document. This is most common in large documents with many nested elements. If the author inadvertently leaves out a closing tag in the middle of a long chain of nested elements, FCKEditor will often re-write the nested relationships in a completely different way than the author intended, cascading down through the document, rewriting almost every element in order to make everything conform - which results in a large amount of work trying to figure out what happened, and then fixing it manually in the Source view - or worse - requiring the author to use a tool like Dreamweaver to correct the errors and get back to what they originally intended. In my personal experience, I had to spend several hours fixing a complex table layout that had been completely munged by this feature.

• It makes troubleshooting bad code very difficult. The usual method of troubleshooting is to look through the source code for the reason for the error - make a change, then check to see the result - rinse, repeat. If the editor is re-writing the code every time it is saved, it really gets in the way of the troubleshooting process, since you never know if the problem is due to the change you just made, or they way that the editor is trying to fix the mistake. It is like having two programmers trying to debug the code at the same time.

• Not all sites are XHTML based. This makes HTML 4.01-based sites non valid.

• Entering Smarty, Javascript, and other types of non-XHTML tags is difficult. Theoretically, you can use regular expressions and the FCKConfig.ProtectedSource variable, but this is difficult and awkward, and we have not had complete success with this in all of our situations. Currently, we are still struggling with how to properly escape all Smarty tags.

• Deprecating the option to toggle the feature on and off reduces choices for us - why reduce options? More options makes the software more flexible.

The feature is definitely an interesting one, and I am sure a lot of work was put into it. However, as it stands, it causes more problems than it cures, and making it optional would be a great help.

In order to make it usable for our purposes, we would need something along the lines of this: Instead of automatically re-writing the code, if FCKEditor could highlight to the user the errors in the source code - perhaps by applying a red color to malformed code in the Source view - this would be much more useful. A method such as this simply provides information to the user, instead of making assumptions and performing actions that the user does not desire. Another idea is to create a dialog interface when the code is not valid. A simple method would be to create an alert pop-up that states, "This code is not valide (X)HTML. Are you sure you want to continue? Click Cancel to return and edit your code."

In any case the automatic re-writing of code should always be only an option.

Thank you!

• Jeff

[Alfonso Martínez de Lizarrondo]

Replying to jfohl:

• Automatic conversion is unintuitive. This is probably the number one issue. Most users are not aware that this is happening. When they toggle from the Source view to theWYSIWYG view, and then back - their code has been modified, and they don't know why. Worse - they sometimes do not know that it has occurred and only see that the result is not what they had intended.

But the browser itself does change the loaded contents in order to parse them. As I said in the forums. Try to load this page in your browser (no matter which one, all of them will change the contents) and tell me if the provided source matches the content of the body as it's seen after the browser has parsed it:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
	"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
	<title>HTML</title>
	<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"  >
	<script type="text/javascript">
		window.onload=function() {alert(document.body.innerHTML)};
	</script>
</head>
<body>
<p>
	Hello:<br />
	Today....
</p>
<table style="background:#00F; border:1px solid red">
	<tr>
		<td>1</td><td>2</td>
	</tr>
</table>
</body>
</html>

• The Cascade Effect. When the FCKEditor attempts to fix errors, it sometimes will re-write the entire document. This is most common in large documents with many nested elements. If the author inadvertently leaves out a closing tag in the middle of a long chain of nested elements, FCKEditor will often re-write the nested relationships in a completely different way than the author intended, cascading down through the document, rewriting almost every element in order to make everything conform - which results in a large amount of work trying to figure out what happened, and then fixing it manually in the Source view - or worse - requiring the author to use a tool like Dreamweaver to correct the errors and get back to what they originally intended. In my personal experience, I had to spend several hours fixing a complex table layout that had been completely munged by this feature.

See above: if you provide invalid HTML the browser will fix it in order to be able to understand what you have created. Test this other testcase, and IE, Firefox and Opera will fix the HTML in different ways:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
	"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
	<title>HTML</title>
	<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"  >
	<script type="text/javascript">
		window.onload=function() {alert(document.body.innerHTML)};
	</script>
</head>
<body>
<p>
	Hello:<br />
	<hr>inside p
</p>
<a href="#">
<table>
		<td>1<td>2
	</tr>
</a>
</body>
</html>

• Not all sites are XHTML based. This makes HTML 4.01-based sites non valid.

As I asked in the forums, could you please provide an example of some content generated by FCKeditor that does validate as XHTML and doesn't validate as HTML?

• Entering Smarty, Javascript, and other types of non-XHTML tags is difficult. Theoretically, you can use regular expressions and the FCKConfig.ProtectedSource variable, but this is difficult and awkward, and we have not had complete success with this in all of our situations. Currently, we are still struggling with how to properly escape all Smarty tags.

I don't know anything about Smarty, but Javascript tags are protected while in WYSIWYG. But that's a different problem that just generating XHTML code.

• Deprecating the option to toggle the feature on and off reduces choices for us - why reduce options? More options makes the software more flexible.

If two options do the same, why confuse people?

In order to make it usable for our purposes, we would need something along the lines of this: Instead of automatically re-writing the code, if FCKEditor could highlight to the user the errors in the source code - perhaps by applying a red color to malformed code in the Source view - this would be much more useful. A method such as this simply provides information to the user, instead of making assumptions and performing actions that the user does not desire. Another idea is to create a dialog interface when the code is not valid. A simple method would be to create an alert pop-up that states, "This code is not valide (X)HTML. Are you sure you want to continue? Click Cancel to return and edit your code."

That means creating a HTML validator that does check the source before switching to WYSIWYG, and probably the easiest way is with an AJAX call to the server and run HTML Tidy there.

[Jeff Fohl]

Thank you again for your timely response. I am sorry I did not continue the discussion in the forums. I did not know you had responded there. I suppose I need to turn on email notification for the forums. Please let me know if you would like to continue the discussion there, rather than here in the ticket. Here are my responses.

But the browser itself does change the loaded contents in order to parse them. As I said in the forums. Try to load this page in your browser (no matter which one, all of them will change the contents) and tell me if the provided source matches the content of the body as it's seen after the browser has parsed it:

See above: if you provide invalid HTML the browser will fix it in order to be able to understand what you have created. Test this other testcase, and IE, Firefox and Opera will fix the HTML in different ways:

I understand that user agents will try to figure out how to best render (X)HTML with syntactic errors - and the parser will rewrite the code internally in order to display it. However user agents won't actually re-write these changes and save over the document that it read. This is essentially what the FCKEditor is doing, which is unintuitive to users. Users expect their code - be it totally valid or invalid to remain intact.

Perhaps I am unclear on exactly how the FCKEditor works. Are you saying that the FCKEditor simply uses the browser's parser to read the code entered - and there is therefore no way to preserve the code unchanged?

As I asked in the forums, could you please provide an example of some content generated by FCKeditor that does validate as XHTML and doesn't validate as HTML?

I remember testing this about a year ago, and finding that the (X)HTML would not validate as HTML 4.01. However, I can't reproduce this problem now - so I may be wrong on this point.

If two options do the same, why confuse people?

I am not sure what you mean by this statement. I fear that perhaps I am totally confused as to what the FCKConfig.EnableXHTML and FCKConfig.EnableSourceXHTML controlled. My understanding is that they controlled whether or not the source code would be automatically rewritten in the source view and WYSIWYG view - forcing all code to be XHTML. Is this not the case? If it is the case, what two options are you referring to? Are you saying that there is another variable in the FCKEditor that accomplishes the same thing?

Thank you. The FCKEditor is a great product despite my complaints!

[Alfonso Martínez de Lizarrondo]

Replying to jfohl:

Perhaps I am unclear on exactly how the FCKEditor works. Are you saying that the FCKEditor simply uses the browser's parser to read the code entered - and there is therefore no way to preserve the code unchanged?

Yes, that's right. FCKeditor uses the browser capabilities to edit html pages, and on top of that it does add the interface, and lots of other code to try to make everything work but the flow basically is this:

HTML -> FCKeditor -> Browser -> now the user changes things, sometimes FCKeditor will interact but others is just the browser the one that handles the keystrokes, drag&drop etc... -> To output now the HTML, FCKeditor has to query the browser to know the current document, so even if the user didn't change anything in WYSIWYG the browser did parse the code.

[Jeff Fohl]

Interesting. So, depending on what browser you are using, you will get slightly different saved code then - correct?

It would still be extremely useful to have a way to prevent this - so that the code is simply a text field that is saved as a variable, instead of running it through the parser.

The idea is to make the source code behave like source code in any other environment. When a programmer runs their code through a compiler, they don't expect to lose all of the source code and have to disassemble it on the other end to make edits.

There really is no way around this?

My understanding of FCKConfig.EnableXHTML and FCKConfig.EnableSourceXHTML is that they accomplished this somehow - as I have seen earlier users write about in various forums, such as here: ​http://www.web-relevant.com/blogs/cfobjective/index.cfm?mode=entry&entry=23CF4EED-BDB9-5320-E1CE7D885D20E5B8

Is that an incorrect understanding? If so - what was the purpose of those variables?

[Alfonso Martínez de Lizarrondo]

Replying to jfohl:

Interesting. So, depending on what browser you are using, you will get slightly different saved code then - correct?

Yes. That why there are bugs that only affect one browser (or one browser version), some bugs can't be fixed, and why there's still no full support for Opera or Safari.

It would still be extremely useful to have a way to prevent this - so that the code is simply a text field that is saved as a variable, instead of running it through the parser.

In order to use the WYSIWYG editing mode the browser parser must be called.

My understanding of FCKConfig.EnableXHTML and FCKConfig.EnableSourceXHTML is that they accomplished this somehow - as I have seen earlier users write about in various forums, such as here: ​http://www.web-relevant.com/blogs/cfobjective/index.cfm?mode=entry&entry=23CF4EED-BDB9-5320-E1CE7D885D20E5B8

Is that an incorrect understanding? If so - what was the purpose of those variables?

Without the use of those settings the editor did use the body.innerHTML value, that as I've stated, will still use the browser parser. The difference is that the output is mostly generated by the browser and now the DOM tree is traversed in order to try to generate the same output (as long as it's possible) in every browser.

If you try to use those settings in some older version when they were available you'll find that indeed it does solves that problem with that specific code in IE 6, but in Firefox it gets distorted anyway due to the internal differences of each browser. So as soon as a single user edits the content with Firefox you'll get everything broken (and nobody can say what will happen with future IE versions). And I've no doubt that although that code did work in IE, it's very easy to have some other code that will be broken as easily as it was in Firefox.

The solution is to use the ProtectedSource capabilities, that way your tags will remain intact.

[Jeff Fohl]

Thanks for the extensive explanation.

The ProtectedSource solution won't work - because it hides all markup. It effectively disables the WYSIWYG mode. In that case, a simple text field would be just as useful.

I am going to try and figure out a method for getting what I want - I have some ideas, but I don't want to waste your time here. Who knows - maybe I will make a code contribution some day... :)

Thank you.

[Frederico Caldeira Knabben]

As precisely explained by Alfonso, output manipulation is needed by the editor. This is a intrinsic part of the editor code.

[Jeff Fohl]

Thank you Frederico.

Yes - Alfonso was very patient with me, and I now have a much deeper understanding of the issue. I am contemplating experimenting with informational displays to warn and guide users when their code is malformed - thus giving them the opportunity to fix their source code before saving or switching to the WYSIWYG view. This would at least be less confusing to the users, and not mystify them when they enter code, preview in WYSIWYG, and then toggle back to Source only to find that their code has completely changed. Let me know if you would be interested in anything that I can do in this area. What I am imagining is something along the lines of an alert box in combination with highlighting malformed code in the Source view.

Thank you both very much for you time.

• Jeff

[Frederico Caldeira Knabben]

Replying to jfohl:

Let me know if you would be interested in anything that I can do in this area. What I am imagining is something along the lines of an alert box in combination with highlighting malformed code in the Source view.

If you come out with a nice solution for it, it is always a good idea to publish it in some way (well... a license requirement actually :)). Just create a dedicated ticket for it (don't use this one).