Home > shell > Utility to non-destructively fold or re-indent HTML

Utility to non-destructively fold or re-indent HTML

September 5Hits:1
Advertisement

I have a script that takes input from wget or similar and searches through it for key words using grep. (I promise i am not trying to parse HTML with regular expressions, it is just a convenient way to emulate the content-detection behaviour we have in another much more complex product.) This works great, as long as the HTML content isn't too severely minified. When it is, the lines can become very long (over 50 kB in some cases i've seen), and grep chokes on them.

To remedy this, i would like to be able to fold or re-indent the HTML so that it is spread out over more lines. However, in order for the script to give accurate results, i need to be able to do this without otherwise altering the content. This means it can't correct invalid or unclosed tags, and it must fold only between elements, not inside them.

These two requirements seem to rule out all of the HTML-tidying or prettifying utilities i've found.

Are there any UNIX-based shell utilities, perl/python/ruby modules, or similar that can do this for me?

Alternatively, since all i need is to add some new lines in between tags, is there a way that i can semi-reliably do this myself?

Answers

Run the file through HTML Tidy.

For example:

curl http://superuser.com | tidy -i | less

-i is for indentation of the input.

Ok, for anyone else in need of this, I'm recording the suggestions made in this awesome thread (in case that link goes down, as per StackExchange guidelines):

  • HTB 2.0 - DOS based - http://www.digital-mines.com/htb/
  • Tabifier - supports CSS, HTML and C style syntax (including Javascript) - http://tools.arantius.com/tabifier
  • HTML-Kit - a full-featured free HTML editor running on Windows, you need to config TIDY options [Tools /Check code using Tidy /Add new config], uncheck all swithes except "Output only the body content" and "Convert non-breaking space to entities", then go to Actions /Tools /HTML Tidy /Indent Tags or beautify - http://www.chami.com/html-kit/
  • SCREEM - only for Linux -
  • NetBeans - " After openining an html file with NetBeans, click Source then select Format. That's it. " -
  • WebmasterGate's HTML / XHTML Beautifier - Online tool - http://www.webmastergate.com/html-beautifier/
  • Aptana Studio (Version 2.0.4) - "Select Edit > Format or press Ctrl-Shift F to format the html code. The format function can be configured from Windows > Preferrences, then select Aptana > Editors > HTML > Formatting, click Edit to add tags which should not take a new line then save it as a new preferrence." -
  • UniversalIndentGUI - Uses HTB Beautifier internally - While running Notepad++, go to Plugins > Plugin Manager > Show Plugin Manager, select UniversalIndentGUI from the available list to install it.
  • tidy with these options:

(filler text since the markdown engine seems to have problem when code directly follows bullets)

[HTML, XHTML, XML Options]
anchor-as-name:no
doctype:omit
drop-empty-paras:no
fix-backslash:no
fix-bad-comments:no
fix-uri:no
input-xml:yes
join-styles:no
lower-literals:no
preserve-entities:yes
quote-ampersand:no
quote-nbsp:no

[Diagnostics Options]
show-warnings:no

[Pretty Print Options]
indent:yes
indent-spaces:3
tab-size:3

[Miscellaneous Options]
quiet:yes

I'm yet to try out these options (the input-xml: yes and force-output: yes config suggestions to HTML tidy mentioned http://stackoverflow.com/questions/7151180/use-html-tidy-to-just-indent-html-code works for my immediate purpose), will update this answer if I do.

The simplest way to do this without parsing/fixing the document is to look for a closing tag, followed by an open-angle-bracket or whitespace, and insert a newline. Search for:

(</[^>]+>)(<|\s)

and replace with

$1\n$2

You will still need to manually check over each output document and verify that it didn't break anything, but this should work for most cases. It won't be pretty output, but it should kill 50KB lines.

Related Articles

  • Utility to non-destructively fold or re-indent HTMLSeptember 5

    I have a script that takes input from wget or similar and searches through it for key words using grep. (I promise i am not trying to parse HTML with regular expressions, it is just a convenient way to emulate the content-detection behaviour we have

  • A text editor or viewer with folding based on indentation

    A text editor or viewer with folding based on indentationAugust 3

    I'm trying to get a better understanding of the *.dsl files as disassembled by the iasl program. Those files looks like: /* * a comment */ Stuff { More Stuff { Beer (a, b) // another comment } Hmm (mmm) { Package (0x04) { 0xFFFF }, Package (0x04) { 0

  • Utility to view how much space folders are taking in your hard disk?

    Utility to view how much space folders are taking in your hard disk? October 26

    Possible Duplicate: Best program to visualize file system usage on Windows? Is there any free utility that will allow you to see how much space each folder is taking in your hard disk? --------------Solutions------------- You may find the WinDirStat

  • Disk Utility for MacAugust 5

    I love my MBP, but I constantly hear the disk spinning up and down. I can't find something similar to smartctl in linux to monitor my load cycles. I don't want to burn my disk out in a year. Can someone recommend an alternative? --------------Solutio

  • Setting up the texmf folder in MacTeX February 4

    I've searched online, but I have not found any clear instructions how to permanently add .sty files to my TeXShop installation. I'm using a Mac with OSX (TeXShop is a mac only program). What is the best way of doing this? The one comprehensive set of

  • Rename folder using wildcard windowsMarch 9

    I've got a piece of software that includes its own backup utility, but it generates folder names with the current date and time appended. I'd like to write batch script that will remove the date/time part of the folder name essentially I'd like to re

  • How do I create a Folder Action Script to tar items dropped in folder?

    How do I create a Folder Action Script to tar items dropped in folder?August 22

    I would like to drop/copy files/folders and have them compressed then copied to my desktop. This is what I know so far... and it isn't much so any help would be appreciated. on adding folder items to theFolder after receiving theAddedItems repeat wit

  • VIM: folding bullet points

    VIM: folding bullet pointsDecember 14

    I'm quite new to VIM, but have been trying to teach myself over the past week, yet I'm encountering a problem which has proven to be quite frustrating: Currently I'm mostly using VIM to take notes, and use the folding capabilty of VIM to make my work

  • Outlook for Mac 2011: Lost a Folder/SubfoldersFebruary 28

    Here's the scenario: After Outlook forced me to rebuild my Main Identity about 6 weeks ago, I found I was missing a mail folder and its subfolders under "On my computer." I immediately did some research on how to recover it, especially from Time

  • Folder Lock in Ubuntu and Windows August 13

    This question already has an answer here: How to encrypt individual folders? 5 answers Is there any way i can password protect or lock folder in ubuntu that can also be opened in windows 8?? I am using windows 8 and ubuntu 14.04 --------------Solutio

  • serving UTFgrid information made with mb-util with leafletJanuary 1

    I've created the mbtiles file with tilemill and exported it on my ubuntu webfolder with the following command: mb-util --scheme=xyz /folder/tile.mbtiles /var/www/html/my-tiles To get utfgrid information in my leaflet map itself I used danzels utfgrid

  • CHKDSK stop (no response) at stage 2 (verifying indexes)July 15

    I am using Windows 7. I was trying to synchronize the folder. Then the FreeFileSync freeze. I think there is some problem to my folder. Then I tried to copy the folder to another location. When calculating the size, the copying dialog freeze. So I su

  • Should I include a selfdestruct-method to my applications?September 19

    I recently had a negative experience, where client bailed from the bill, but my middle-man already uploaded our software and design to the clients server. Client turned out be a known criminal, and of course he changed all possible passwords of the s

  • What is the necessary file structure for a DVD to be playable?October 15

    The backstory: I've purchased a DVD via online download (from EZTakes.com). The files appear in this kind of directory tree: DVD Name +-- VIDEO_TS/ | +-- (various video files) +-- cover/ | +-- (a couple of .jpgs of the DVD cover art) +-- content.info

  • Problem with Macbook air automatically not acquiring free wifi network dns or router address

    Problem with Macbook air automatically not acquiring free wifi network dns or router addressNovember 1

    I have this problem when my macbook air sometimes does not connect to free wifi hotspots. When the problem happens and I try to run the diagnostics, it normally gets to yellow network settings tab. It seems that it for some reason does not acquire au

  • Can Exchange 2007 auto-reply to an incoming message?February 26

    Can Exchange 2007 auto-reply to an incoming message? I don't want to set it up in Outlook, I want the reply to be done on the server. --------------Solutions------------- Out of Office auto-replies are setup within Exchange, and reply to any incoming

  • Problems running some apps on mac, no error messageMarch 4

    I hope this question is within superuser scope. I've recently downloaded an application that just happens to be a computer game but this is not pertinent to the problem. Thing is, I can't run any of them on my mac, and I have no idea why. Whenever I

  • How to merge (and not replace) folders when copying on the Mac?

    How to merge (and not replace) folders when copying on the Mac?March 8

    There's a similar question about windows. This is the same, but for mac. By the way, it's odd there's no question on this yet, since it seems like it's the most hated thing on Mac OS X. If I try to copy or move a folder to somewhere it already exists

  • How to stop Time Machine on Mac use of removable disk?June 1

    One of my friend recently bought a Mac and somehow when she connect her removable disk to the computer, Time Machine took control of this device uses it as backup device automatically -- she could not use the disk for other purpose any more. When we

  • Where can I hide my SSID and password protect it in AirPort Express?

    Where can I hide my SSID and password protect it in AirPort Express?August 10

    For school I have to configure my airport express to their requirements, I know how to disable DHCP on my mac, but from there how do I hide my SSID and password protect it? --------------Solutions------------- To change the settings on an Airport (Ex

Copyright (C) 2017 ceus-now.com, All Rights Reserved. webmaster#ceus-now.com 14 q. 0.472 s.