PROOF Series #1 — Compliance Coding Rubric

Document version: 1.0 — Drafted May 14, 2026 Author: Shawn Lai, WTT Digital Purpose: Operationalize California Rules of Professional Conduct 7.1 and 7.4 into a scored, reproducible categorical scheme for law-firm web content.

What this rubric is for

The PROOF Series #1 study asks whether material California Rule 7.1 compliance correlates with AI-search citation rates. To answer that question rigorously, "material compliance" has to be operationalized as something more precise than a subjective read. Two independent coders applying this rubric to the same site should arrive at substantially the same score.

This document is the rubric. It is published as a standalone artifact so that the methodology can be audited, replicated, or challenged before data collection begins. Pre-registration of the analysis plan that uses this rubric is published separately.

Scope of the rubric

The rubric applies to public-facing law firm web content in the following surfaces, in this order of priority:

1. The homepage of the firm's primary domain
2. The "About" / "Our Firm" page
3. Each named practice-area page
4. Each named attorney bio page
5. The meta description and Open Graph tags of each of the above

It does NOT cover:

• Off-site content (Avvo profiles, Justia listings, JD Supra bylines) — these have their own platform-specific rules and are out of scope for this study's compliance scoring
• Paid advertising content (Google Ads, Meta Ads) — separate Cal Bar rules apply
• Email content — typically not public-facing
• Social media content — separate study would be required
• PDF or downloadable assets — too variable across firms to code consistently

The rubric is built for California firms. Firms licensed in multiple states are scored against California rules only; multi-state extensions are noted but not scored.

The five flag categories

Each surface listed above is scored on five flag categories. Each flag is binary (present / absent) but weighted differently based on Cal Bar enforcement priority and AI-citation impact (per the working hypothesis).

Flag 1: Specialization / specialist claims (weight: 3)

Trigger phrases (case-insensitive, partial match counts): - "specialist" / "specialists" - "specializing in" / "specialize in" / "specializes in" - "specialization" - "specialty"

Rule basis: California Rule of Professional Conduct 7.4(c) reserves the words "specialist," "specialize," and related forms for attorneys formally certified by the State Bar's Legal Specialization program.

Permitted exceptions (no flag if): - The phrase appears with explicit reference to a State Bar Certified Legal Specialist designation (e.g., "Certified Specialist in Family Law, California State Bar Board of Legal Specialization") AND the attorney holds that certification - The phrase appears in a direct quotation from a third party (e.g., a court ruling, a news article) and is clearly attributed - The phrase appears in the firm's name itself (rare but possible for grandfathered firm names)

Scoring: 1 point per surface where the trigger appears outside the exception, up to a max of 5 points per firm (homepage + about + three practice-area pages). Weighted 3x. Maximum contribution: 15 points.

Flag 2: Superlative quality claims (weight: 2)

Trigger phrases: - "best" (when describing the firm, its attorneys, or its services) - "top" / "top-rated" / "top-tier" - "leading" / "premier" / "preeminent" - "most experienced" / "most respected" / "most trusted" - "award-winning" (without proximate award identification) - "exceptional" / "outstanding" / "unparalleled" - "elite" - "world-class"

Rule basis: California Rule 7.1 Comment [3] flags language likely to create "unjustified expectation." Florida Rule 4-7.13 (used as a stricter comparative since California firms with multi-state practices often face Florida exposure) explicitly prohibits superlatives without verifiable third-party rating disclosed inline.

Permitted exceptions (no flag if): - The superlative is supported by a named third-party rating in the same paragraph or within 100 words (e.g., "named a Super Lawyers Top 100 attorney in California, 2024-2025") - The superlative appears in a direct quoted client review (testimonials have their own disclaimer requirements but the superlative itself is not the firm's claim) - The superlative refers to a measurable, verifiable attribute (e.g., "the most settlements above $1M in San Francisco County in 2024" with the underlying data citable)

Scoring: 1 point per superlative instance, up to a max of 10 points per firm. Weighted 2x. Maximum contribution: 20 points.

Flag 3: Predictive outcome language (weight: 3)

Trigger phrases: - "guarantee" / "guaranteed" / "we guarantee" - "we will get you" / "we'll get you" - "we will recover" / "we'll recover" - "successful outcome" / "successful result" (predictive) - "we ensure" / "we fight to ensure" - "winning result" (predictive) - "you will" (in any predictive context — "you will receive," "you will recover") - "count on a [positive outcome]"

Rule basis: Cal Rule 7.1 Comment [3] directly addresses language likely to create unjustified expectation about results. This is the highest-risk category because it most cleanly fits the rule's stated trigger.

Permitted exceptions (no flag if): - The phrase appears in a past-tense factual statement (e.g., "We recovered $2.4M for our client" — this is a past-results claim, scored under Flag 4 instead) - The phrase is part of a generic ethical commitment ("we guarantee our commitment to client communication" — describes process, not outcome) - The phrase appears in a quote attributed to a client testimonial (separate scoring path)

Scoring: 1 point per instance, up to a max of 8 points per firm. Weighted 3x. Maximum contribution: 24 points.

Flag 4: Past results without proximate disclaimer (weight: 2)

Trigger: - Specific monetary recoveries or settlements stated by amount (e.g., "$2.8M jury verdict," "recovered $1.2M for our client") - Specific case outcome descriptions (e.g., "won dismissal," "prevailed at trial," "successfully defended") - Phrases summarizing patterns of outcomes (e.g., "successfully recovered substantial amounts," "track record of favorable verdicts") - Named precedential cases the firm worked on (when used to imply future capability)

Rule basis: Cal Rule 7.1 Comment [3] plus the standard Cal Bar past-results disclaimer requirement.

Required disclaimer language (any of the following counts as adequate): - "Past results do not guarantee a similar outcome." - "Past results do not guarantee future outcomes; each case is evaluated on its own facts." - Substantially equivalent language conveying the same meaning

Disclaimer placement requirements: - Same page as the past-results reference - Within 200 words of the reference, OR in a clearly visible footer/sidebar that applies to all past-results content - Same surface (a disclaimer in a separate "Disclaimer" link page does NOT count — the disclaimer must be reachable in the same view)

Scoring: 1 point per past-results reference without an adequate proximate disclaimer, up to 8 points per firm. Weighted 2x. Maximum contribution: 16 points.

Flag 5: Fee and free-consultation language without required qualifiers (weight: 1)

Trigger phrases: - "No fee unless we win" / "no fees unless we WIN" / variations - "Free consultation" (when paired with personal injury or contingency-fee practice) - "Contingency fee" (when used without clarifying client cost responsibility) - "No upfront fees"

Rule basis: California Rule 7.4(d) requires that contingency-fee communications disclose that the client may remain responsible for costs and expenses regardless of outcome. The phrase "no fee unless we win" without that disclosure is technically non-compliant.

Required disclaimer language: - Some form of: "Client may be responsible for costs and expenses regardless of outcome." OR - "Costs and expenses are separate from attorney's fees."

Disclaimer placement: Same page, proximate (within the same content block or a clearly linked footer on the same page).

Scoring: 1 point per instance without proximate qualifier, up to 5 points per firm. Weighted 1x. Maximum contribution: 5 points.

Aggregate scoring

Total possible flag points: 15 + 20 + 24 + 16 + 5 = 80 points maximum.

A firm with zero flagged content scores 0. A firm with significant exposure across all five categories approaches 80.

Classification cutoffs for the study

For the PROOF Series #1 analysis, firms are classified into three bins:

Cutoffs are chosen so that Bin A captures firms with at most isolated borderline issues (e.g., one unqualified superlative) and Bin C captures firms with patterned issues across multiple categories. Bin B is the gray zone and is reported separately but not used as the primary comparison group.

The primary study comparison is Bin A vs. Bin C for citation-rate analysis. The Bin B middle is reported descriptively but not used to test the central hypothesis, because the categorical contrast is what gives the hypothesis test its power.

Coder protocol

To ensure inter-rater reliability:

1. Two independent coders score each firm. No discussion until both have completed scoring.

2. Each coder reviews the five surfaces in the documented order (homepage → about → practice-area pages → bios → meta tags). Scoring happens surface by surface, not by flag category — the coder applies all five flag categories to each surface before moving to the next.

3. Coders use a structured spreadsheet with one row per (firm, surface, flag) tuple. The aggregate firm score is calculated, not assigned.

4. Disagreements resolved by adjudication. Where the two coders' scores differ, a third coder reviews and either ratifies one of the two scores or assigns a new score. The third-coder score is final. All disagreement instances are logged for the methodology appendix.

5. Inter-rater reliability is reported. Cohen's kappa across coder pairs is calculated and reported in the final paper. Pre-study target: kappa ≥ 0.70 (substantial agreement). If pilot kappa falls below 0.65, the rubric is revised before full-study coding begins.

Adapting the rubric for other states

This rubric is California-specific. Adaptations to other state rules would adjust:

• Florida (Rule 4-7.13): Flag 2 (superlatives) weighting increases; testimonials require explicit handling; pre-filing review requirement is referenced but not scored (pre-filing is process, not content)
• New York (Rule 7.1): Flag 4 (past results) disclaimer language becomes more specific ("Prior results do not guarantee a similar outcome" must appear in the same communication, verbatim)
• Texas (Disciplinary Rule 7.02): Flag 3 (predictive outcomes) weighting increases; the rule's "unjustified expectation" trigger is enforced more actively than in California

For the present study, California-only scoring keeps the methodology clean. Multi-state extensions are future work.

Limitations of this rubric

The rubric does not capture:

• Implicit testimonial language ("clients consistently report...") that implies a testimonial without quoting one. This is a real Cal Bar issue but coding it consistently requires more interpretive judgment than the rubric's structure allows. Future versions may incorporate.
• Image and video content. A firm video saying "we guarantee results" violates the same rules but is not in scope for this rubric, which scores text only.
• Hidden content. Content behind logins, paywalled assets, or interactive widgets that don't render statically is not scored.
• Dynamic content variation. A site that randomizes its hero text or A/B tests headlines may present different content to different visitors. The coder uses a single snapshot per surface and notes any obvious variation; longitudinal variation is out of scope.

These limitations are acknowledged so the paper's claims stay calibrated to what was actually measured.

Version history

• v1.0 (May 14, 2026): Initial published version. To be referenced by the PROOF Series #1 pre-registration document.

Future versions will be tracked here. Substantive changes after data collection begins require a methodology update and a re-coding pass.

This rubric is part of the PROOF Series #1 study materials. The full pre-registration document is at wtt.digital/proof/cal-bar-citation-study/pre-registration. Methodology questions, replication interest, or peer-review feedback can be sent to [email protected].