Comments on: Automatic Index Recommendations in PostgreSQL using pg_qualstats and hypopg https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/ Fri, 22 Dec 2023 00:58:37 +0000 hourly 1 https://wordpress.org/?v=6.5.2 By: Me yea me https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10972557 Wed, 29 Jul 2020 20:33:23 +0000 https://www.percona.com/blog/?p=58654#comment-10972557 To anyone reading this in the future. That view pg_qualstats_indexes doesn’t seem to exist anymore. Now you just query the function directly:

select * from pg_qualstats_index_advisor(); to get the magic sauce you’re looking for 🙂

The fn can take params I suggest checking the readme: https://github.com/powa-team/pg_qualstats/

]]>
By: Me yea me https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10972556 Wed, 29 Jul 2020 20:11:40 +0000 https://www.percona.com/blog/?p=58654#comment-10972556 Same problem. Did you ever get it solved??

]]>
By: Me yea me https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10972555 Wed, 29 Jul 2020 20:11:02 +0000 https://www.percona.com/blog/?p=58654#comment-10972555 I got as far as running the sysbench-tpcc but then afterwards trying to query pg_qualstats_indexes — it doesn’t exist in my db.

I can query pg_qualstats and pg_qualstats_pretty etc etc, but pg_qualstats_indexes does not exist in the db. When is it supposed to be created?

]]>
By: Alexander Nikitin https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10972252 Wed, 20 May 2020 08:44:58 +0000 https://www.percona.com/blog/?p=58654#comment-10972252 Hello! I repeated what is described in this article, but could not find the view pg_qualstats_indexes. There are only the following relationships: pg_qualstats, pg_qualstats_pretty, pg_qualstats_all and pg_qualstats_by_query (select relname from pg_class where relname like ‘pg_qualstats%’;)
Versions: PostgreSQL 11.8, pg_qualstats 2.0.1
What am i doing wrong?

]]>
By: jflambert https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10971989 Wed, 25 Mar 2020 15:02:25 +0000 https://www.percona.com/blog/?p=58654#comment-10971989 Thanks Lazy DBA. I noticed the function doesn’t work either. Hopefully the author fixes his article.

]]>
By: LazyDBA247 https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10971788 Sun, 09 Feb 2020 10:17:49 +0000 https://www.percona.com/blog/?p=58654#comment-10971788 Please remove:

and qs.queryid not in (-2862297032983545665)


my bad..

]]>
By: LazyDBA247 https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10971787 Sun, 09 Feb 2020 08:43:30 +0000 https://www.percona.com/blog/?p=58654#comment-10971787 need to update the function find_usable_indexes() to support columns with case sensitive names

fixed version:
CREATE OR REPLACE FUNCTION find_usable_indexes()
RETURNS VOID AS
$$
DECLARE
l_queries record;
l_querytext text;
l_idx_def text;
l_bef_exp text;
l_after_exp text;
hypo_idx record;
l_attr record;
/* l_err int; */
BEGIN
CREATE TABLE IF NOT EXISTS public.idx_recommendations (queryid bigint,
query text, current_plan jsonb, recmnded_index text , hypo_plan jsonb);
FOR l_queries IN
SELECT t.relid, t.relname, t.queryid, t.attnames, t.attnums,
pg_qualstats_example_query(t.queryid) as query
FROM
(
SELECT qs.relid::regclass AS relname, qs.relid AS relid, qs.queryid,
string_agg(DISTINCT quote_ident(attnames.attnames),’,’) AS attnames, qs.attnums
FROM pg_qualstats_all qs
JOIN pg_qualstats q ON q.queryid = qs.queryid
JOIN pg_stat_statements ps ON q.queryid = ps.queryid
JOIN pg_amop amop ON amop.amopopr = qs.opno
JOIN pg_am ON amop.amopmethod = pg_am.oid,
LATERAL
(
SELECT pg_attribute.attname AS attnames
FROM pg_attribute
JOIN unnest(qs.attnums) a(a) ON a.a = pg_attribute.attnum
AND pg_attribute.attrelid = qs.relid
ORDER BY pg_attribute.attnum) attnames,
LATERAL unnest(qs.attnums) attnum(attnum)
WHERE NOT
(
EXISTS
(
SELECT 1
FROM pg_index i
WHERE i.indrelid = qs.relid AND
(arraycontains((i.indkey::integer[])[0:array_length(qs.attnums, 1) – 1],
qs.attnums::integer[]) OR arraycontains(qs.attnums::integer[],
(i.indkey::integer[])[0:array_length(i.indkey, 1) + 1]) AND i.indisunique)))
and qs.queryid not in (-2862297032983545665)
GROUP BY qs.relid, qs.queryid, qs.qualnodeid, qs.attnums) t
GROUP BY t.relid, t.relname, t.queryid, t.attnames, t.attnums

LOOP
/* RAISE NOTICE ‘% : is queryid’,l_queries.queryid; */
execute ‘explain (FORMAT JSON) ‘||l_queries.query INTO l_bef_exp;
execute ‘select hypopg_reset()’;
execute ‘SELECT indexrelid,indexname FROM hypopg_create_index(”CREATE INDEX on ‘||quote_ident(l_queries.relname::text)||'(‘||l_queries.attnames||’)”)’ INTO hypo_idx;
execute ‘explain (FORMAT JSON) ‘||l_queries.query INTO l_after_exp;
execute ‘select hypopg_get_indexdef(‘||hypo_idx.indexrelid||’)’ INTO l_idx_def;
INSERT INTO public.idx_recommendations (queryid,query,current_plan,recmnded_index,hypo_plan)
VALUES (l_queries.queryid,l_querytext,l_bef_exp::jsonb,l_idx_def,l_after_exp::jsonb);
END LOOP;
execute ‘select hypopg_reset()’;
END;
$$ LANGUAGE plpgsql;

]]>
By: Avinash Vallarapu https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10971033 Thu, 25 Jul 2019 17:47:06 +0000 https://www.percona.com/blog/?p=58654#comment-10971033 This logic only checks if the index can decrease the total cost of execution by running an EXPLAIN. If the optimizer picks up the index, it means that the index can improve the query. Before creating that index in reality, you must manually check any other side effects of creating that index. Today, this logic does not check any other aspects except for the one criteria – is it being considered by the optimizer or not.

]]>
By: James Finnerty https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10971032 Thu, 25 Jul 2019 10:29:15 +0000 https://www.percona.com/blog/?p=58654#comment-10971032 Adding a secondary index on a frequently-updated column that was not previously indexed could disable the HOT optimization and could have very negative consequences on fragmentation and performance. Do the index recommendations take the effect on HOT optimization into account?

]]>
By: Ramesh https://www.percona.com/blog/automatic-index-recommendations-in-postgresql-using-pg_qualstats-and-hypopg/#comment-10971022 Mon, 22 Jul 2019 17:09:15 +0000 https://www.percona.com/blog/?p=58654#comment-10971022 Wow.. this is excellent.. thanks Percona and Avinash.
Keep doing the great work

]]>