User talk:Polygnotus/DuplicateReferences

Linking to this page

Are you meant to link to this page from main pages: Someone like You (Adele song) Shouldn't it link to a Wikipedia rule? Anthony2106 (talk) 07:21, 28 November 2024 (UTC)

@Anthony2106 Sorry I am not sure what you mean. Polygnotus (talk) 10:37, 12 July 2025 (UTC)

@Polygnotus well templates on pages nomerly link to wikipedia rules not userspace pages. e.g. autistic masking has a thingy at the top that goes to a wiki rule, not a user page. Anthony2106 (talk) 14:43, 12 July 2025 (UTC)

I think the user is referring to the template {{More medical citations needed}} which displays the words "add the appropriate references" which is a wikilink that links right back to the article the template is inserted into, instead of linking perhaps to WP:RS or a better choice for the wikilink.   ▶ I am Grorp ◀ 15:05, 12 July 2025 (UTC)

Deduplicate exact clones

If two citations are exact (not partial) duplicates, it would be great if this script could automatically deduplicate them. – Closed Limelike Curves (talk) 23:01, 1 December 2024 (UTC)

Exactly, otherwise it encourages drive by tagging when fixing would have been pretty easy. AncientWalrus (talk) 09:08, 25 January 2025 (UTC)

@Closed Limelike Curves and AncientWalrus: Thanks! True, but I decided to make another script for that, which isn't really finished yet. It kinda works, but it ain't pretty. See User:Polygnotus/Scripts/DeduplicateReferences. Polygnotus (talk) 10:03, 12 July 2025 (UTC)

Has User:Polygnotus/Scripts/DeduplicateReferences finished beta testing? rootsmusic (talk) 16:17, 30 December 2025 (UTC)

Not sure if it is officially, but I've been using it for a while now and I think it's great! I haven't come across any major issues with it. – Closed Limelike Curves (talk) 22:33, 30 December 2025 (UTC)

@Rootsmusic I haven't "officially" released it but as @Closed Limelike Curves states it should work just fine. I'll add "officially" releasing it to my todolist. Happy new year! Polygnotus (talk) 15:44, 31 December 2025 (UTC)

False positives in script results

Hello. If this is not the proper place to share these comments, please tell me where to re-direct them.

I am not a programmer, but I am a librarian who has lots of experience reading and writing citations of all sorts and with an attention to detail. I first stumbled across the results of your script when I was fixing CS1 problems on the article Timeline of African-American firsts. The output of your script found in this article is dated September 2024.

Summary: I would like to respectfully recommend that your script not report duplicates based solely on ISSN. In the examples cited below, every instance of a citation reported as a duplicate that contains an ISSN in the original ref is in fact NOT a duplicate citation. The original citations in your report that did not contain an ISSN are in fact duplicates and should be consolidated. By design, the ISSN is a standard number assigned to a serial (periodical) at the work/title level; individual articles that contain a ref with an ISSN are not duplicates simply because they were published in the same periodical. (This is not usually true of ISBN, because those should be unique at the work/book level.)

Examples: In the Timeline... article, the script claimed that there were three urls that were considered duplicates:

This article contains several duplicated citations. The reason given is: DuplicateReferences detected:

https://search.worldcat.org/issn/0021-5996 (refs: 137, 138, 212, 219)

https://msa.maryland.gov/msa/mdmanual/08conoff/ltgov/former/html/msa13921.html (refs: 293, 296)

https://www.nytimes.com/2020/07/27/us/politics/john-lewis-memorial.html (refs: 345, 352)

It is recommended to use named references to consolidate citations that are used multiple times.(September 2024) (Learn how and when to remove this message)

______

As of today (January 9, 2025), the second example (msa.maryland) was determined to be a true duplicate and has already been consolidated by another editor. Those citations did not include an ISSN value in the original references.

The third example (nytimes.com) is also a true duplicate, and I intend to consolidate it after getting feedback to my comments. Those citations also did not include an ISSN value in the original references.

The problem is with the first example. The ISSN 0021-5996 is assigned to Jet magazine, and the four citations listed all do contain this value within the ISSN parameter in the original refs. (I noticed that your script appears to be doing some normalization on the ISSN field, because the url that begins https://search.worldcat.org/issn/ is not found within the original references at all.) But none of the Jet citations are actually the same article; the only bibliographic parameter in these citations that is duplicated is the ISSN. These citations should definitely not be consolidated with each other.

I could say more about my experience trying to fix this one example, but I'll leave it as is for now. Thanks for considering a change to your script to make it possible for editors to avoid investing time in reviewing a potential duplicate according to your script's output, when it is really is a false positive. IMHO. NOLA1982 (talk) 19:26, 9 January 2025 (UTC)

Came here to say exactly what the poster above said.Naraht (talk) 19:22, 29 April 2025 (UTC)

@NOLA1982 and Naraht: Sorry for the delay I didn't have this page watchlisted for some reason.

The script should skip all ISSNs, because it should skip all links that start with https://search.worldcat.org/issn/

Is this problem still occuring for you? If so, can you give an example of an article on which you see this problem? Polygnotus (talk) 10:37, 12 July 2025 (UTC)

Hello. Sorry that I can't offer more examples at this time. I've not come across this output in a while.

But are you saying that you have already addressed the problem with script revisions? If so, thank you! NOLA1982 (talk) 19:55, 13 July 2025 (UTC)

Appears to be a similar problem at Sudanese civil war (2023–present), this edit manually removed dup cites, and then this edit a day later added the tag. AFAICT there's no dups, it's just that they have archive urls.

Personally I'm not a fan of this, tags are very ugly and take up important space, something as trivial as dup cites which only really impact the aesthetics of the references section isn't worth them imo. Think people should use this tool to identify them, and then fix all the ones they can. Advice along those lines on this page basically warning against pure WP:DRIVEBY would be good Kowal2701 (talk) 20:56, 26 July 2025 (UTC)

Thank you!

This is incredibly useful! I see there's a concern about false positives, and that inexperienced editors might misuse the tool, but for me the simple ability to highlight possibles is very helpful. Valereee (talk) 11:40, 20 January 2025 (UTC)

@Valereee Thank you! If you notice any false positives please let me know. Polygnotus (talk) 10:01, 12 July 2025 (UTC)

Will do! Valereee (talk) 10:38, 12 July 2025 (UTC)

Drive by tagging

This script seems to encourage drive by tagging which is discouraged when fixing is straightforward. Arguably, fixing duplicate refs is straightforward. AncientWalrus (talk) 09:09, 25 January 2025 (UTC)

Sadly, fixing duplicate refs is, in some cases, far from straightforward, especially when dealing with pagenumbers and quotes. The good news is that they are working on a sub-referencing feature. Polygnotus (talk) 09:50, 12 July 2025 (UTC)

Extraneous line breaks in bulleted list (tagging function)

I love this script, however the tagging function adds an extraneous <br> at the end of a bulleted item. I have noticed that some editors remove these br's (rather than fix duplicate citations). Is the br necessary, or could it be removed from the function that adds a tag at the top of the article? Two recent examples of an editor removing the line breaks: [1] [2]   ▶ I am Grorp ◀ 15:14, 11 May 2025 (UTC)

@Grorp Thanks, see here. Polygnotus (talk) 09:54, 12 July 2025 (UTC)

Works like a charm, now. Thanks, Polygnotus!   ▶ I am Grorp ◀ 14:32, 12 July 2025 (UTC)

Template placement

Please consider checking whether this tool leads editors to place maintenance templates in the wrong place at the beginning of articles. Review this edit, for example. Maintenance templates should be placed under hatnotes and protection templates, per WP:ORDER. Thank you 155.190.8.5 (talk) 21:28, 18 June 2025 (UTC)

Hiya! It has been upgraded to use the MoreBits placement logic. Maybe that is imperfect. I'll investigate.

I think MOS:ORDER should be coded into JS and AWB, but no one has so far. Polygnotus (talk) 10:00, 12 July 2025 (UTC)

Wikipedia:Database reports/Articles containing links to the user space

I think the script link from the Duplicated citations template parameter should be removed, as it populates the Wikipedia:Database reports/Articles containing links to the user space page. This makes the report less useful. You could create a project space redirect to your user script and use that instead. Just a friendly suggestion. – DreamRimmer ■ 09:43, 12 July 2025 (UTC)

@Polygnotus – DreamRimmer ■ 09:43, 12 July 2025 (UTC)

@DreamRimmer This better? Polygnotus (talk) 09:48, 12 July 2025 (UTC)

Looks good. Thanks :) – DreamRimmer ■ 09:53, 12 July 2025 (UTC)

Thanks for pinging I didn't have this page watchlisted for some reason. Polygnotus (talk) 09:53, 12 July 2025 (UTC)

Not working as well

This script doesn't seem to be working for me lately. I get a error message saying " Failed to add the template. See console for details." What console? Mox Eden (talk) 09:34, 26 July 2025 (UTC)

what console?

browser console https://firefox-source-docs.mozilla.org/devtools-user/browser_console/index.html

here is the error i got, might be different than yours

Error: ReferenceError: Morebits is not defined. addDuplicateCitationsTemplate index.php:118 Lokiretro (talk) 21:08, 8 August 2025 (UTC)

@Lokiretro On which version of Wikipedia was that? Was it here on the English Wikipedia? Polygnotus (talk) 22:18, 8 August 2025 (UTC)

english version Lokiretro (talk) 22:20, 8 August 2025 (UTC)

Template placement still wrong

@Polygnotus: This script was just used by Sink Cat to place a garish template at the top of a prominent article on the main page: Tom Stoppard. The placement at the top of the article seems contrary to the documentation which says that it Adds a "add {{Duplicated citations}}" link to the References section header. The issue of duplicate references is a minor one and so keeping the template in the References section or the article's talk page seems sensible. Why is the script not doing this? Andrew🐉(talk) 21:00, 4 December 2025 (UTC)

@Andrew Davidson: Thing is, some people find the placement at the top of the article the only correct one, and others the one below the 'References' section header. IIRC the people who designed the current housestyle of templates decided to put everything up top, even for example when it was about categories at the bottom of the page. Another group says that things should be in the most relevant location (so they would place the category template all the way at the bottom of the page). To be honest, neither is a perfect solution. The underlying problem is, as you noted, that the template is huge and garish. That is perhaps a feature, not a bug, when it is superduper important (e.g. an article has been edited by a COI editor) although we also have a page that says "no disclaimers". At some point I did have a conversation about it with the one and only @GreenLipstickLesbian:, see here. Because there are many such templates fixing one of them just leads to a discussion about the next one, and there are hundreds. The reduced size of the templates I proposed would fix much of this problem in my view. Gimme a sec I'll take a look. Polygnotus (talk) 21:15, 4 December 2025 (UTC)

I also thought about not allowing the template on pages with over x pageviews if I remember correctly. Polygnotus (talk) 21:22, 4 December 2025 (UTC)

Thanks for responding but I'm still not understanding why the script doesn't do what its documentation says it does. Is this a bug, inaccurate documentation, an undocumented change or what, please? Andrew🐉(talk) 21:25, 4 December 2025 (UTC)

@Andrew Davidson Because someone else had the opposite opinion, and I didn't update the documentation. One sec. Polygnotus (talk) 21:27, 4 December 2025 (UTC)

@Andrew Davidson User:Polygnotus/DuplicateReferences4.js is a thing of great beauty. But it would be nice if the community could actually decide where they want to place such boxes. See MOS:ORDER. Please ping me if you propose a change. Polygnotus (talk) 08:43, 5 December 2025 (UTC)

See also Wikipedia:Huge message boxes. Polygnotus (talk) 09:25, 5 December 2025 (UTC)

Hey @Andrew Davidson, sorry that my edit was disruptive! I noticed that there were duplicate citations and wasn't able to rectify them at that time, so I believed that it'd be helpful to point them out to other editors so they could fix the issue. If it's a nuisance in general, I'll refrain from placing the template going forward. — Your local Sink Cat (The Sink). 21:51, 4 December 2025 (UTC)

Seems not to be working recently?

I've used DuplicateReferences successfully for a while, with

importScript('User:Polygnotus/DuplicateReferences.js')

in common.js (copied, no typo). But for a few weeks I've had the impression that it's no longer working, perhaps due to a change in Wikipedia, or something I've done. I never get a list of duplicates at the start of references, even when there are clear duplicates. Best wishes, Pol098 (talk) 14:36, 10 January 2026 (UTC)

• @Polygnotus: It's also not working for me, including on the test page referenced in the instructions. I tried uninstalling and reinstalling but I still do not see the table that I am supposed to see at the top of the references section. Z1720 (talk) 22:58, 10 January 2026 (UTC)

• • @Polygnotus:@Z1720:It's reported as having last been edited on 10 January 2026. In a single test I did (Boris Johnson) it found several duplicates, as before. It presumably went wrong and has presumably been repaired; if so, many thanks. Pol098 (talk) 18:40, 11 January 2026 (UTC)

Hey there. I've noticed that this script doesn't seem to want to work. When opening up the console, I get the lines

[DuplicateReferences] URL: https://en.wikipedia.org/wiki/User:Polygnotus/DuplicateReferencesTest index.php:11:25 [DuplicateReferences] Reflist div not found, script terminated

This occurs on all articles even with duplicate citations present.— Your local Sink Cat (The Sink). 04:51, 13 January 2026 (UTC)

@Pol098, Z1720, and Sink Cat: Someone made a change elsewhere which broke the script. Should work now. Thanks! Polygnotus (talk) 05:24, 13 January 2026 (UTC)

@Polygnotus Thanks for the fix! — Your local Sink Cat (The Sink). 09:10, 14 January 2026 (UTC)

Multiple lists

Can you fix the script so that it only generates a single list with multiple items? Multiple lists don't make syntactic sense and cause various problems. See MOS:ACCESSLIST. Here is an example: Multiple lists added here [3] and fixed here [4]. GA-RT-22 (talk) 15:57, 21 January 2026 (UTC)

@GA-RT-22 What do you think about this? Polygnotus (talk) 18:03, 21 January 2026 (UTC)

You're asking the wrong guy! Looks like it should work but I have not tested it. GA-RT-22 (talk) 19:44, 21 January 2026 (UTC)

Script not detecting differences between references

Hi, this seems like a useful script, but it seems a bit overzealous. An editor ran this script today on Outer Wilds, and FA, and tagged that there were 4 duplicates across 13 refs. The problem is that none of the four were duplicates. Three of them were to videos (hosted on youtube) where each reference had a different "time" value, and the fourth was 2 refs to a book with 2 different chapters. Can the script be updated to check a level deeper more than just "has the same url" or "has the same isbn"? --PresN 23:25, 6 March 2026 (UTC)

@PresN Hmm lemme investigate one sec. Polygnotus (talk) 23:33, 6 March 2026 (UTC)

@PresN Good point, thank you! I tried fixing it without adding too much extra complexity. This should work, please let me know if you see any other edge cases/unexpected behaviour. Polygnotus (talk) 00:13, 7 March 2026 (UTC)

Awesome, thank you! --PresN 02:44, 7 March 2026 (UTC)

{{duplicate citations}} consensus says put at References section

Do change the script for DuplicateCitations, as the consensus for the duplicated citations template is to put it in the References section instead of the top of the article. Qwertyxp2000 (talk | contribs) 11:37, 25 March 2026 (UTC)

@GreenLipstickLesbian Would you be so kind to change MOS:ORDER and the template documentation? Thanks, Polygnotus (talk) 03:06, 27 March 2026 (UTC)

!!! MOS edits, edit warring over the MOS, long grudges over the exact formatting of a template, site ban, eventually arbcom ban, here I come!!! GreenLipstickLesbian💌🧸 03:32, 27 March 2026 (UTC)

@GreenLipstickLesbian Woohooo! New LTA!!!!!!!!! Polygnotus (talk) 09:12, 27 March 2026 (UTC)

Please write a ballad in my honour, and sing it at my SPI. GreenLipstickLesbian💌🧸 14:43, 27 March 2026 (UTC)

@GreenLipstickLesbian [5] It is actually very appropriate to cry while singing Blood is Just a Darker Shade of Pink (Bedazzled Edition). It enhances the sound. Polygnotus (talk) 18:41, 27 March 2026 (UTC)

Just deleted an inaccurate template from your scrip

Please see the edit history at Canadian Confederation. Mr Serjeant Buzfuz (talk) 05:54, 6 July 2026 (UTC)