Is the order preserved?

Yes. The first occurrence of each line stays where it was; later repeats are dropped. Sorting is a separate endpoint.

Is the comparison case sensitive?

Yes. Two lines that differ only in case are two different lines, because in identifiers and emails that distinction sometimes matters.

What about trailing whitespace?

A line with a trailing space is a different line. Run fix-spacing first if you want those treated as duplicates.

Remove duplicate lines API

Strip duplicate lines from any list, log, or CSV while preserving the original order.

Get a free token Full API reference

Endpoint	POST https://astraltext.com/api/v1/remove-duplicate-lines
Auth	Bearer token, free
Sends	input: text
Returns	result: text
Price	Free. 1,000 calls a day, 60 a minute.

When to call it

Merged exports, concatenated logs and pasted lists all carry repeats. Deduplicating in a pipeline keeps the count honest before the data reaches whatever will act on it once per row.

For a one-off, the browser version needs no token and never sends your data anywhere: it runs on your own machine.

Example

curl

curl -X POST https://astraltext.com/api/v1/remove-duplicate-lines \
  -H "Authorization: Bearer ast_YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input": "ada@example.com\ngrace@example.org\nada@example.com\nalan@example.net\ngrace@example.org"}'

response

{
  "tool": "remove-duplicate-lines",
  "result": "ada@example.com\ngrace@example.org\nalan@example.net",
  "chars": 50,
  "ms": 1
}

In your language

JavaScript

const res = await fetch("https://astraltext.com/api/v1/remove-duplicate-lines", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.ASTRAL_TOKEN}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({"input": "ada@example.com\ngrace@example.org\nada@example.com\nalan@example.net\ngrace@example.org"}),
});
if (!res.ok) throw new Error((await res.json()).detail);
const { result } = await res.json();

Python

import os, requests

res = requests.post(
    "https://astraltext.com/api/v1/remove-duplicate-lines",
    headers={"Authorization": f"Bearer {os.environ['ASTRAL_TOKEN']}"},
    json={"input": "ada@example.com\ngrace@example.org\nada@example.com\nalan@example.net\ngrace@example.org"},
    timeout=30,
)
res.raise_for_status()
result = res.json()["result"]

Parameters

Field	Type	Notes
input	string	Required. Up to 200,000 characters per call.

Errors

Errors are RFC 7807 problem documents with a stable type you can branch on.

missing-token	401	No Authorization header.
bad-request	400	input missing or the wrong type.
tool-failed	422	The input could not be processed. detail says why.
quota-exceeded	429	Past 1,000 calls today.

Questions

Is the order preserved?: Yes. The first occurrence of each line stays where it was; later repeats are dropped. Sorting is a separate endpoint.
Is the comparison case sensitive?: Yes. Two lines that differ only in case are two different lines, because in identifiers and emails that distinction sometimes matters.
What about trailing whitespace?: A line with a trailing space is a different line. Run fix-spacing first if you want those treated as duplicates.

Other endpoints

A token takes thirty seconds

Free, no card, 1,000 calls a day, and the same token works on astraltext.com, astralpdf.com, astraljson.com and astralbatch.com.

Get a free token